The Great eTrain Robbery? (Please Opine)
September 7, 2009
After re-reading and pondering my last post, I believe I was intentionally vague enough to prevent getting exactly the feedback I need. For those of you who already commented on that previous post, let me apologize for not asking the question in the way I should have asked it. I’m going to try again.
The Powers That Be at my agency have recently signed a large contract with an eLearning development vendor. There are several courses included in this contract, one of which has been assigned to me to manage.
The particular course in question is approximately 2 hours of classroom soft skills training that needs to be delivered in an eLearning format. The content has already been written for the classroom. It needs to be repurposed for eLearning. The course will be developed using a Lectora-style system that produces what is essentially an HTML/javascript page turner. Multimedia (animation, narration, etc.) will be minimal. The course will not be narrated in its entirety, but there may be some snippets of narration here and there. Interactions should be basic form-based questions created within the development application. Graphics will include basic stock images/clip art in the classic “eLearning that looks like a bad PowerPoint presentation” style.
The fixed-price contract that has been signed with the vendor for this course is for 766 hours of development at an average hourly rate of $116 for a total of $89,000+.
This is only one of six courses in this contract, which totals $717,000+.
Because I work for a government agency and these are public funds being spent, I’m morally obligated to watch for (and report) wasteful spending. Particularly for projects to which I am personally assigned. (Please note that I had no input on the solicitation or contract process itself.)
I need to be sure I’m on solid ground before I put myself in the sure-to-be-very-unpleasant position of being the one to say “Hey, wait a minute!”
And that, esteemed colleague, is where you come in.
Is this price – given the basic course parameters I provided – outrageous, questionable, or acceptable? How many hours would you expect to spend on a project of this nature (though my details are admittedly sparse)? Or how much would you expect to pay an outside vendor?
Please share your thoughts in the comments section below. This may be your money we’re talking about.
Posted in eLearning Design
content rss
September 7th, 2009 at 9:10 am
Hi Chris:
As I have said, I’m not a professional in this field but I work every day in the design of materials and courses for e-learning delivey and, in my opinion, both the price and the hours are overestimated, no doubt. 776 hours for a 2 hours course? 89,000$ is an overpriced from any point of view (I teach Public Finance to my students, and we know about public expenditure…)
Perhaps the opinion from anyone more professional could help you more than mine.
Good luck,
Pedro.
September 7th, 2009 at 10:09 am
I too think this is overpriced. I’m an independent contractor who has been developing eLearning applications for 15 years for a wide variety of industries and organizations, including the Federal government. While the contractor’s hourly rate is higher than mine, it is not exceptionally high. What I question is the number of hours of development. Even if they used the rule of thumb of 200 development hours per finished hour of training, the project would only require 400 hours. And that rule of thumb includes some subject matter research, video creation (which is very often included in soft skills training), rich interactivity, etc. Even with full narration, a page turner with stock media and little interactivity would likely fall in the 100 development hours (or less) per finished hour range.
Hope that offers some guidance…
Steve
September 7th, 2009 at 12:04 pm
No official way I came up with this, but I would think it would be more like a 100 hour project (and that’s be very generous). You never know how you will have to repurpose the content from stand up to e-learning.
I think the price you stated is outrageous!
September 7th, 2009 at 12:28 pm
Chris,
I have been building elearing for about 10 years now. Even in the beginning when you had to have an instructional designer, a web designer/graphic artist, and a programmer to turn out anything decent, this price would have been on the high end. This price is outrageously high. So let me explain. The $116/hr isn’t that bad. Generally you are looking at $80 – $150 depending on the complexity. However 776 hours to develop the course is insane. With the content already developed it should take you 30 – 80 hrs. Granted when you are working with the government you have to adhere to additional rules and regulations which does cost additional money, but this is a ridiculous.
September 7th, 2009 at 12:54 pm
Higher education institutions frequently hire adjuncts to design online courses for less than 1/10 of this, albeit using course management software that is already licensed to the institution.
Find a large university near you and look into any online offerings they might have. You’d be amazed at how cheaply they designed some of these courses!
September 7th, 2009 at 4:30 pm
I am not an expert and maybe I can’t manage the figures in US dollars, but I know how much it is overpriced are the internet services specifically in the Education field. I know there is a lot of work in the construction of a good service of e-learning, but there is a lot of open source resources we can use to build a good course. What are the reasons to spend a lot of money in a training course when there are a lot of alternatives to take? Money talks.
September 7th, 2009 at 10:23 pm
I have been involved in providing custom eLearning solutions for 9 years. $89k for a 2 hour course (of Lectora kind with minimum multimedia) is certainly VERY high. 766 development hours is outrageous. I would imagine that to be between 80 & 200 hours depending on how structured/complete the content is, availability of SME, process adopted etc. The hourly rate of $116 too is high, specially for this job. Not sure what the constraints are in this case, but outsourcing/offshoring part of the development work could get the average rates down to less than half.
I agree with a comment above that using CMS you could possibly do the job for almost 1/10th of this.
September 8th, 2009 at 8:06 am
Steve,
The rule of thumb in the industry is that eLearning development effort is in a range from about 40 hours per hour of instruction (for the page turners, with no pre-existing content as in your case) to about 800 hours per hour of instruction (for advanced simulations with moderate multimedia production values). High-end video games can go as high as 3,000 hours — but that’s really a different application and a different market.
So, yes, you’re being overcharged.
September 8th, 2009 at 1:08 pm
Without seeing the contract and requirements, it’s hard to say if it’s too high. However, based on what you’ve shared, my assumption is yes. Fortunately, you’ll recoup a lot of that investment at the other end come April 15.
I think the learning opportunity here is to figure out how people get stuck in these contracts. part of the problem is that the organization’s don’t always understand what is and isn’t difficult to do with the elearning tools. So they take a hourly wage like $100/hr and then use a ratio like 300:1 (for elearning dev) and then figure that at about $600K for two hours of training is reasonable.
What might make an interesting post is ten tips on how to avoid this in your organization. I know that at a previous place we had some groups that would go outside for their development. I think they assumed that we didn’t have the resources. In those times, we tried to intervene and at least be part of the review process if we didn’t get the work. It always annoyed me because we were able to do the work at about 30% of the cost since our income and time was already factored in.
September 8th, 2009 at 3:17 pm
@ Tom (and all):
One of the most frustrating aspects is that internally, eLearning development is often looked upon as brainless work for low-level peons with low-level salaries – see http://www.equixotic.com/2008/08/25/the-value-of-good-elearning-developers/ – yet in a vendor contract those $120-180 per hour rates look just fine and dandy.
Can someone please explain to me the obvious contradiction?
At the very least, I can use this ridiculous contract to argue the value of my salary as well as the inordinate amount of time I’m going to need to do that simple page-turner conversion. “A 2-hour classroom course you say? No problem! I can get that converted to a lame page-turner for you in only…5 months!”
I can’t wait to see how *that* flies…
September 8th, 2009 at 3:48 pm
I hear ya. At a previous place I would talk about different ways to use wikis for our training and they’d look at me like I was an idiot. But then two weeks later, Masie would mention using wikis and they’d send a link to me saying we should consider using wikis.
If you’re internal, you’re not as good as a vendor.
I’d take the figures you have and calculate all of your training at the same rate. Then at the end of the year show how you build x hours of training that would have cost $1.3 million, but since they only paid you $50K, they saved a lot of money.
September 8th, 2009 at 3:56 pm
@ Tom:
I tried using the “We’re paying more for a single 2-hour course conversion than you pay me in a year!” line with my manager. Her response? “Don’t forget to add your benefits.”
*facepalm*
September 9th, 2009 at 8:42 am
Chris,
Speaking as someone who’s spent the last 8 years designing & developing a variety of eLearning for large multinationals, as well as a few of those years managing and sourcing dev teams and vendors for some of those projects…HOW CAN I GET IN ON THIS??? =)
Seriously, the amount is SO high (by at LEAST 5X I’d say, and probably 10+) that it blows away my initial hesitancy to give an opinion on a project about which I don’t know many details. Even factoring in any Section 508 compliance that may be required since it’s a gov-funded project… and that will probably be taken care of by the dev tool anyhow. Is the vendor being required to take on some bizarre sort of limitless liability for the actual *content* somehow? (I doubt it, but it’s all I can think of…!)
WOW.
September 10th, 2009 at 10:48 am
Chris,
As Karl Kapp mentioned over on Tony Karrer’s blog, I think the bigger issue is that the entire $89,000 (and perhaps the whole $770,000) is just being thrown out the door. There is no way that a ’soft skill’ program ported over into ‘elearning that looks like bad PowerPoint’ is going to have ANY lasting effect. The organization may as well just flush that money down the toilet.
The ONLY thing it will do is reduce travel and production expenses for the ILT course it is replacing. If that is the primary driver of this project, then the organization should just cancel the training completely and put $770,000 back in the bank. They would be better off doing nothing than putting the workforce through hours of ‘click next to continue.’
In order to obtain real performance improvement, it’s likely that the programs would need to be completely redesigned to take advantage of the online medium.
The evidence is out there. What drives decision makers to think this type of product will have any benefit?
John
September 10th, 2009 at 12:53 pm
I agree with Andy. Those numbers seem to be unbelievable. I recently did a couple localization conversions for a friend’s elearning company, and while the work ended up being a bit more than the pay, they weren’t any more complex necessarily than your work appears. The cost was more on the order of $10-15k (my guestimate) for one course for that client. Now the work was totally different – translating content and so on – but the amount of time it took was definitely more than it appears this one course you are managing will take. $89,000 sounds like way too much for a page turner. And 766 hours of development? Maybe if you were starting from scratch and the project would take 4-6 months, but that’s nowhere near the case.
BTW, thanks for doing the due diligence on this.
September 10th, 2009 at 1:25 pm
@ John:
I agree 100% with your “what’s the point?” argument. I am vehemently opposed to trite page turner courses like this (and this opposition forms the foundation of my blog), and I agree that they’re a complete waste of time and money, for both the developers and the learners.
But that’s another battle I’m fighting on another front.
September 11th, 2009 at 4:29 am
Outrageous. Brandon Hall surveyed 200 companies and came up with a ratio of 33:1 for PowerPoint to eLearning conversion. That gives us a development time of 66 hours and suggests that this vendor is charging more than 10 times the industry standard. Based on the information provided here, I would say that the taxpayers are getting ripped off.
September 16th, 2009 at 1:51 pm
While the numbers seem high, there is insufficient information (imo) to judge this. “Soft skills” training is broad and this offers no insight to the impact of what this eLearning is intended for (would negotiating with terrorists be a soft skill?).
The hourly rate is fine and maybe on the low side (again, imo); it seems that the number of hours is the more critical aspect.
Brandon Hall published an estimate (as stated by Ben above) of time to develop eLearning here and this may provide you with a bit more of an empirical look at this particular eLearning contract.
34:1 Instructor-Led Training (ILT), including design, lesson plans, handouts, PowerPoint slides, etc.
33:1 PowerPoint to E-Learning Conversion.
220:1 Standard e-learning which includes presentation, audio, some video, test questions, and 20% interactivity
345:1 Time it takes for online learning publishers to design, create, test and package 3rd party courseware
750:1 Simulations from scratch. Creating highly interactive content
December 26th, 2009 at 11:36 pm
My company regularly does e-learning courses that include podcasts, a virtual library of supplemental reading materials, wiki, forum and on-site workshops in computer labs to meet participants and trouble-shoot any problems. We also include a pre-test and post-test to assess effectiveness. In short, we do a great deal more than just create a course. The last contract we had to develop six, twelve-hour courses was around $450,000 – but it include much more than just the courses. So, in short, $717,000 is well more than double, probably triple, what you could pay another company to do the same work.
January 20th, 2010 at 1:22 pm
The rate per hour seems to be inline, but the development hours are somewhat inflated for what you describe. I would not expect the development time to take more than 500 hours for the development of the 2 hours of e-learning.
At my company we take about 240 manhours to develop a 20 minute intractive sofatware tutorial using Captivate.
August 24th, 2010 at 9:19 am
I’m late to the party, but for basic rapid eLearning like you’ve described, I consistently see about 35:1 bantered about.
I work on a large government software simulation contract and we propose 450:1, due to the high levels of customization (Flash simulations of real software), changing requirements, unstable/changing technology, etc. I think the base level is somewhere around 320:1 for highly sophisticated simulation (guidance provided by the customer’s governing documents about training development times) with extra-time for the instability of the program/software as outlined above.
And your labor rate is pretty average from my experience (10 years private contracting with government/military clients).