Lesson Learned: Hidden Requirements

I spent four months in the Discovery Phase of a major project.

It had some of the most complex business rules I'd ever run into. The idea was to build a data platform to move the company's data from third party vendors into a custom data warehouse. That's where it started. As you'd expect, the scope blossomed like a mushroom cloud in 1950s Yucca Flats, Utah.

I concentrated on the original concept since others were dealing with the side projects. I created a matrix of coupon-mailer-product and took a week to write some pseudo code to demonstrate the machine logic required (it wouldn't fit into a single Use Case- the best I could do is add my 15 pages of 8 point pseudo code to the 53 pages of the rules and walk the developer through it). The SMEs approved it. It was like pulling teeth. You'd get smiles and words of assent, but some of the folks I was dealing with were playing company politics and pretty much ignored the Use Cases for their multi-million dollar data application/framework.

About a week before I was scheduled to leave (real old school - using Iterative Methods). I chanced to talk to a director about something else in the Smoker's Lounge. Nice guy, very knowledgeable. He politely asked me how we were coming and I outlined what I'd done to date. "You mean you're not creating coupons when xxxxxx occurs? We've been doing that by hand for years."

Turns out there were twice as many business rules and decision trees as I had.

The rules and logic were in someone's desk drawer. He thought we had a copy. Even though I'd asked him for the rules at least four times. "Yeah, you mean I forgot to mail it to you?"

I left the project three days later. Blamed for the failure.

The poor development team would have to do it from the original  material which had been amended about four or five gazillion times since it was written in 1856.

Here's what I should have done and have done ever since in Waterfall and Iterative Methodologies:

1. Identify each and every stakeholder and find the possible politics that can impact the project. Make sure it's in the Risk Management Plan with mitigation strategies. No seriously. Doing that would have saved the first phase of the project.
2. Identify every document passed to me (I was swimming in that Project from Hell, but I didn't catalog each piece of documentation). I could have identified gaps with a simple spreadsheet.
3. Draw more pictures, tables, graphs and diagrams. I used to think a multi-million dollar project was probably worth reading the Use Cases. Silly me. Pictures. Color. Arrows. Boxes. The third or fourth rule of technical communication: make sure there's at least one graphic in each publication and if the document is in color, it's more important than one in black and white. You think I'm kidding, doncha? I'm not. Its true- ask any tech writer.
4. Always, always look for evidence of hidden requirements. And then look again.
   
5. Keep asking polite questions and reflecting what the SME says back to him or her until I'm certain I understand what s/he is saying.
   

This project is the reason I love Agile Development Methods. There are no hidden requirements in an Agile Project. When you deliver working code the the SMEs, Stakeholders and end users , they can play with it and tell you what's wrong and what they think they need. And the politics/See-what-we-can-get-fer-free mentality is eliminated- the business is on the team and controlling each sprint. We don't point fingers, we find causes and improve our quality. There are no long and boring Use Cases. Instead there are 3-4 line User Stories from which I can create a Technical Specification for the developer to use in coding. Everybody gets to weigh in on problems and ideas for improvements to increase quality and increase throughput.

Best of all, we create software without games or gotchas.

Powered by ScribeFire.

No SAP for Me

Well, I thought about it long and hard. I'm giving up on SAP Data Warehousing and Business Intelligence.

I took four lessons on how to do it. I wanted the big-time cash promised in a recession proof endeavor.

So here's what I found out:

• They told us we could really screw-up our client's data by doing the wrong thing. Me? Screw up enterprise data with an inaccurate click somewhere? I have my own problems without adding to them.
• Does it scare anyone else that no one, SAP Engineers included, know everything there is to know about SAP? Sample question supporting the thesis: How do you create and perform boundary, integration and regression tests when you come out with new stuff?
• I get there are four basic import routines for the stuff we were doing. Non-SAP data sources, Flat File and two, count 'em two SAP Data Sources. Hmmm. One for non-SAP and two for SAP sources. Is anyone else confused on how SAP abstracted the import (Extract), cleanse (Transform) and use (Load) processes?
  
• SAP creates words that have nothing to do with function, I'll be generous and say its because the software is written in Germany by ex-IBM engineers (like there aren't any native speakers of English in the U.K. or here in the U.S. that can test terms with users and suggest better ones). I do not believe that just because we did it this way for x number or releases, its the best way to do it. An example (i.e. one that I was able to figure out): There's an object called an InfoProvider. So I spent an hour finding out that an InfoProvider isn't a non-SAP data source (which is still another SAP object) but is an internal SAP Object that provides data to the modeling and reporting tools. Oh yeah, some of the term definitions change between SAP revisions. It's no wonder SAP needs to certify users. As a goof, look up the word 'replicate' in an SAP 7.x version ETL (Extract, Transform and Load) Manual. It's not what you think it is.
• Because of the bullet point above, I was confused. It was like Algebra in High School. It made sense while I was watching, but when I tried to do it, I was lost. Couldn't internalize the stuff. Labeling is important to me and when you start changing definitions willy-nilly, I get rattled. I couldn't tell the difference between the objects. Relational Databases have been using really good terms for a couple of decades now. For what we were doing, SAP abstracts the data and de-normalizes it (data delivery is not the goal- data manipulation is) from the key fields so it can pretty much let you do anything you want. I worked with something similar in a data intensive application where all the data goes into a single, two column table (just like SAP) and it used four other tables to identify, and if required, normalize the data- SAP uses an intermediate table with what's billed as a really cool feature, but is, in actuality nothing more than an automatic, system-assigned GUID (Global Uniform Identifier) table for data look-up and intermediate/abstracting tables and data. I just explained it to you in less than ten sentences with only a couple of jargon words. I guess that's why SAP can charge a gazillion dollars.
  
• What's the most basic data import anyone ever does? Yes. Import a flat file. It took me over a week to figure out how to do it. The book (written by SAP) pretty much had nothing to do with the interface I was using. The latest version of SAP makes certain steps optional when importing data. Not by my sights. I had to create about 123 Objects four or five times each until I could do the import and there's no freakin' way I could replicate what I did and in what order.

Working with bean counters, executives and managers day in and out over the specifics of reports for weeks on end would be torture for me. I've seen others do it and do it well. I have all the respect in the world for them and am glad when they're on my team. I just can't do it. I'm much more interested in team dynamics, technology, helping folks to solve problems, improve processes and stuff like that.

To thine own self be true.

So I found out the State of Illinois might pay for PMP (Project Management Professional, i.e., a certification telling the world I can be what I've been doing for about ten years- be a project manager) and ITIL Foundation(I had to look this up on Wikipedia- it's a framework for software development, like this is hard) AND it'll pay for the test(s) andf first year membership in PMI (Project Management Institute, yeah, sounds dumb to me, too and its a really pain in the a^& to re-certify. Seems like a major gimmick to me). I have an appointment Thursday to begin figuring it out.

I talked to the wife and we both decided this stuff really wasn't in my line, just not a good fit. She's a developer and teacher. She's been trying to teach me .Net stuff for a long time: a. yeah, I can do it, b. it should work the first time I run it and c. stay away from me when I'm debugging, bounds and integration testing. No really. Stay away from  me.

We agreed I'm an IT BA, perhaps there's a decent Project Manager underneath the requirements gathering top veneer, but that's pretty much it. I know how stuff works, I have good communcation skills and can explain it to the business, developers, QA and end users. I can gather requirements, design/mock-up reports and am a pretty good jack-leg Information Architect. And I have a particular propensity for governance and process (in other words, I like systems analysis and I can help others learn different methodologies). And yes, I like using the word propensity. When I was on radio, it took the chief engineer an hour to clean up after me when I said on the air.
 
It was almost as satisfying as saying goodbye to a long tern, 'challenging' client.

I appreciated the chance to try. Right clicking on SAP objects over and over and over would drive me up a wall. I'm already a squirrel, so I see no advantage.

Powered by ScribeFire.

Differentiation Isn't Enough

One of the places I used to work is just about ready to go belly up. I'd always hoped when the economy turned itself around, I could go back there (and I'm still cheer leading for it to succeed). The company treated its people as they wished to be treated and attracted really smart and sharing people. Seriously. They did. That bred a LOT of loyalty in me and I'm sure a lot of the others that have had to leave over the last few months.

I'm not really sure what the business model was. The executives never invited me into the discussions.

But it appeared to me the company was selling itself by differentiating itself from the rest of the boutique development firms based on two things:

• User Experience Design and Testing
• Agile Development Methods

A couple of years ago, I'd have told you that was enough- nobody was doing either. None of the contracting firms I knew about anyway. A few if them would call in a design firm for the "look and feel stuff," but none of us hardy Development Men (strains of Stout Hearted Men should be starting up in your mind right now) really cared much. That was for sissies.

It was really bad for us to think that way, especially with some of the interfaces we came up with. Ugly. Counterproductive. Silly.

But soon, both terms became buzzwords and the economy crapped out. Bad thing, these buzzwords. I remember a manager got his job by spouting buzzwords at about four buzzwords per sentence in his interview- and he got the job. Since I was the guy that had to do all the stuff he talked about in his interview, I quickly learned he knew nothing. I didn't resent it, part of anyone's job is to make your boss look great to his/her boss. I just realized that buzzwords mean nothing.

Today, everybody says they're Agile and say they use appropriate UxD (User Experience Design) standards. And it ain't true.

They don't use Information Architects who create work flows (they call 'em scenarios, whatever) and design for the end user with the latest HMI Design Standards...that's Human-Machine Interface Design. Nor do they test GUIs (Graphical User Interfaces) and ask end users how well something works or doesn't work.

And they've never read the Agile Manifesto, else why is the Project Manager (PM) running the show without any input from team members? Or no test-driven development? Or no parallel programming? Or keeping documentation to a minimum and letting the team run itself with the PM's role changed to removing obstacles (with the BA, of course). Nope. Not Agile. They put silk purses on sows' ears.

So when the company I worked at was trying to drum up business, it got lost in the shuffle because its perceived expertise was allegedly shared by every other company. There was no differentiation.

Frankly, companies pulling in development consultants:

• Don't care about methodologies. They care about results. And its way too easy to oversell Agile's working code paradigm.
• Don't care about the end user much. If they did, they'd have mastered this years ago. The Mac has been out how many years? They should care, because ease of use significantly increases productivity which translates to dollars and cents....but when my former company 'educated' potential clients, we had little impact because of the lack of front-end buy-in. Just do it and write the cost into the bid.
  
• Want the consultants to have the answers. The customer, allegedly, don't have time to supply a team member with enough clout to actually make iteration decisions nor provide a great deal of time as Subject Matter Experts (SMEs) to the BA/Developers...they have to do their jobs you know.

I wrote some marketing white papers. Not once did I mention UxD or Agile. I told the hiring firms about the very real advantages my company offered and how that would impact the budget after implementation. It would have helped a lot if I had some real figures I could use. I don't think they were ever used because our Marketing Director went back to her specialty of media relations.

And some clients are necessarily Waterfall organizations. I worked at one retailer on an application that touched everything in the building and all of the stores. Neither my company nor the customer had a clue about what Agile is or is not. They created a 'War Room' where a lot of IT people sat, but they required Use Cases and High Fidelity Wireframes- I mean, I was using VISIO and PhotoShop to create wire frames. I was told we were using 'Modified Agile." Right. All Agile projects use modified and pragmatic methods. Neither company knew how to scale an Agile Project with Iterative/Waterfall components. So the whole thing blew up in my company's face after it moved back to Waterfall with some minor iterative design work.

Add the distinct lack of domain knowledge for vertical markets (we do Health Care, no- we do hosting, no- we do education, no-we do UxD, etc. etc.) forced the BAs, IAs and Developers to learn new domains for every project. Domain knowledge is the stuff you'd know if you worked five or more years in Insurance or Bakery Goods. It's the business, its regulations/regulators and very basic requirements. For example, I have a lot of domain knowledge about IT, software development, journalism (electronic and paper) and direct marketing (I've worked two projects in that area). The company I'm rooting for is about ready to go belly up because its pretty much bidding on anything that came in the door and has been doing that for several months.

Instead of concentrating on customer's needs and wants then specializing in a particular area, my former company blew it. I hope they get a contract soon so the business won't fold and it could rehire a lot of us.

But I'm not holding my breath.

Powered by ScribeFire.

SAP: The Highs and Lows So Far

So I'm using this blog to increase my visibility for potential employers. If I was them, I'd like to know who I'm getting.

They don't seem to be reading this, so, I'm going to add SAP busisness intelligence/business warehousing (that'd be BI/BW for those of you with TLA [Three Letter Abbreviation] Scorecards out there...and BO (that's be Business Objects without a scorecard) which SAP got a hold of sometime last year and is integrating it.

Or maybe they are and are steering clear?

Anyway, a very smart company is training me to to a. gather the reporting requirements, b. find the data elements somewhere in the spaghetti that is any database and c. actually create the reports. Apparently, this will save the SAP owners a boatload of cash by hiring one person to fill two roles- even by paying us a lot of cash. Us out of work BAs get jobs at what recruiters tell me are obscene hourly rates. I can live with that.

But I still need a short to medium term contract gig to pay bills and make sure I don't lose my house.

Companies (HUGE companies, Billion dollar plus) buy SAP and its various 'modules' (think up-selling, like they do at Radio Shack [Need a microphone with that tape recorder, Mister? How about some cassettes and an extra battery to go with that power supply?]) to run the company. Some former IBM-Germany Software Engineers tried to sell IBM on it in the late 80s or early 90s. IBM took a pass (would you expect less from a company that now sells Lotus Notes, AS400s, r6000s and WebSphere?).

With all that data lying around in DB2 and Oracle databases (think really, really big databases, here), someone got the idea of 'warehousing' the data in one place and generating reports from that data. Enter Data Warehousing. At least three other companies offer it and two others offer reporting and data mining skills. But single, soft serve (pun intended) service from one provider seemed like a great idea. It probably is.

And just what kind of hat does a data miner wear? Hard hat for when the boss starts throwing things? A beanie with a propeller to reinforce the stereotype?

So I've taken two whole days of training so far. I've been taught and think I understand the stupid object names (totally confusing- I need to look up the difference between InfoObject and InfoProvider). So....I figure loading a flat file (think spreadsheet with a half dozen columns and a half dozen rows) ought to be a piece of cake- after all, I've designed and specified this ability along with the required field mapping GUI a dozen times for both .Net and J2EE applications.

It took me three and a half days (seven half days) to figure it out in SAP. And I'm not really sure it's there because we haven't gotten to the report making stuff yet.

Actually, I didn't use SAP. The data warehouse and reporting tools aren't in SAP. And SAP isn't SAP. SAP used to be known as R3 and is, in this latest edition, called ECC. I used is a Java-based tool called NetWeaver. So far, the only reason  I need to know that is to find useless information in the SAP help website. Otherwise, no one seems to use the term.

What I've learned so far:

1. SAP needs someone to come up with better and more intuitive terms and phrasing for this future user. I dunno- maybe have a native speaker of the language eliminate the ambiguity and increase the uniqueness of the object names for starters?
2. SAP is so big and does so much and is so granular that no one knows even everything to know about modules. It supports TWO, count 'em TWO application languages (the proprietary ABAP and Java).
   
3. Connecting a series of optional and mandatory objects probably should defined somewhere, step-by-step, and then explain SAP Module and other database connectors and options.
4. SAP's 'Star Schema' isn't new or unique- I've had Technical Architects design exactly the same thing for data rich environments to speed queries and system response time. It just makes Query creators who know SQL and typical RDBMS (Relational Database Management Systems) crazy.
5. There's a reason anyone with SAP knowledge commands a premium wage.
6. I'm really glad about #2 and #5. I spent way too much time satisfying my ego in Radio News before I got smart and moved into IT...and two lengthy lay-offs (after the NASDAQ crunch and this depression) I just might be able to retire after all.

This is so exciting that I'm almost ready to stop using a very low grade tranquilizer to get to sleep- all I have to do is imagine the NetWeaver interface and creating an InfoCube....zzzzzzzzzzzzzzzzzzzzzz

Powered by ScribeFire.

Agile Job Hunting

I'm getting calls and e-mails from IT professionals with a job opening who think they understand Agile (they don't, but it's the latest buzzword and process) and are asking me questions about it.

It's no wonder the economy's on the rocks. How can you hire someone when you really don't understand what you do and how you do it?

To wit (pun indended):

• One manager at a large company north of me said 'RUP/UML is an Agile Method.' I suggested it is not, but it is a precursor process and is valid unto itself, depending on the circumstances. I should have kept my mouth shut. The manager said, "I went to school about this and it is an Agile Method." I responded (stupidly and because I thought it was a test) the Agile Manifesto demands test-driven development, self-organizing teams, very short sprints, creating usable code from the first and a lot of communication between team members, end users and stakeholders- Iterative does not. I didn't get the job. Shame on me. On the other hand...if I got the job and the manager was that stupid, how satisfying would it have been?
  
• Another company sent me a questionaire. I filled it out. The big thing they wanted to know is when the BA would 'let' developers talk to the business side. I said it depends- in the Scope/Planning Phase, not at all since we need to define the scope of the project, basic architecture and very high level requirements (I like going in with a Technical Architect or Lead Developer and a Project Manager/Scrum Leader if I'm not acting in that role). Later on, it pretty much depends on the developer and if s/he can talk to business people. If they can't, I'd have to facilitate and translate. I didn't get the job. If they have developers talking to Champions and Stakeholders even before the scope is settled, how good could the process be?
  
• One company I worked for, said it was Agile. The organization it was working for was Waterfall. I suggested it was not- they didn't allow the BAs to talk to the developers (!), the daily Stand-up took an hour and allowed yelling back and forth between the customer and the Lead Developer. I saw the mis-match immediately but couldn't find anything quick enough before the ax fell. They called it 'Modified Agile.' Right. I lost the job. With several other folks.
  

Then a couple of weeks ago, a small but very cool start-up advertised for an Agile BA. Instead of a silly technical phone screen- the first thing it wanted was for applicants to create User Stories for a test feature. Finally. I could demonstrate what I could do and they'd actually answer my questions as though it were a real feature. I spent a whole hour on the story and then another hour to create a couple of wireframes and sent 'em in.

A week later, we set up the phone screen! I'd passed the first step.

They had a developer call me. He wanted to know what a BA could do for two developers and a UxD (User Design Experience) professional on a working e-Commerce and Collaboration site. Cool.

I think the interview went well. The company sold me in the ad- they have very smart people working there and I want to be part of it. Next steps are a couple of hairy face-to-face interviews and then a 'team lunch.' My fingers are crossed.

Even if I don't get this one, I have a lot of respect for this company- they think things through and the hiring process actually measures if the person can perform to the level the company needs. Not glad-handing, coming up with silly answers to stupid, predictable and trite questions (which measure your interviewing ability rather than work skill set). Finally! No games. Grown-ups. Adults.

Cool.

Why the hell can't the others learn what the buzzword means and find a good way to hire like minded/trained folks? It's not like its hard...you just gotta think. 

Hmmmm.

Powered by ScribeFire.