by John Toepfer

For most software development projects, both internal and externally sourced, success is not a binary equation.  There are often compromises or partial satisfaction in either scenario.  At the end of the day, the question becomes, how large were the compromises and, taken as a whole, do they equate to a failure of the project?

Over my 25 year career in delivering software solutions to major corporations, I’ve seen both commercial solutions and internal development efforts that have struggled.  My observation, however, is that commercial solutions are much more likely to succeed in the end and to represent the smart and sustainable path.  It’s amazing how often I watch internal development initiatives that either never get off the ground or fail in some crucial way.  In either case, the remedy is to write off that investment and start over shopping for a commercial solution.

As you’re reading, you might argue that my evaluation is biased based on the fact that I own and operate a technology firm. However, if success or failure of an internal technology development effort is measured against the same criteria as a vendor solution, then bias is not a factor.

 

Here are six criteria for evaluating the success of an internal vs. a commercial tech solution:

 

#1 – Timeline & Budget

The question to ask: Can the project be completed in a timely manner and close to budget predictions?

Internal project costs are seldom tightly tracked or cost controlled.  Generally everyone keeps their job and gets their full paycheck for the duration, even if the project goes into triple-overtime or even fails.

Even under poor circumstances, the business has more recourse and cost controls on the vendor solution.  Additionally, to the degree that the commercial solution is based on real product technology and experience, the commercial solution is simply less likely to fail.

#2 – Functionality

The question to ask: Can the right functionality be achieved and is it competitive with commercial solutions?

Generally, there is little incentive for an internal solution to go beyond minimum requirements. With a single client setting the specs and an internal team developing a one-off solution, there is a significant risk that the results will not go beyond minimum functionality and will be well below commercial standards.

Commercial technology solutions can be vetted against their longevity, client references, and competing products.  Furthermore, when the technology deliverable is core to their line of business and industry reputation, the vendor has every incentive to make the investment that keeps the product technology competitive in the marketplace.

#3 – Testing

The question to ask: Can you provide a solution that is  well-tested and bug-free? Will the development team be responsive to bugs and issues encountered by users prior to and during deployment?

The discipline of software testing, particularly if significant user interfaces are involved, is challenging in even the best scenarios.  Doing this for a one-off solution with an informal testing team means that the end user group will be the guinea pigs.

With a larger use-case library, and more established development and testing teams, commercial tools are much easier to test (even when heavily customized for client use).  Again, references can be checked and performance standards can be set and enforced.

#4 – User Documentation

The question to ask: Will the technology be delivered with complete and appropriate end-user documentation and training?

Development of documentation and training material is typically an afterthought for internal projects.  Even when it is accomplished, it’s often not well vetted through repeated use and enhancement. Commercial products and reputable vendors typically have a much more well-developed documentation discipline.  You can even ask to see the documentation before the vendor is chosen.

#5 – System Documentation

The question to ask: Will the system architecture be well-documented such that if persons involved “move on” it can be supported by new resources?

Well produced and thorough technical system documentation is rare with internally developed solutions.  It is often treated as an afterthought and, as it adds cost to a cost center, it is quickly sacrificed to help contain budgets.

Vendor solutions are created on the assumption that there will be a version 2.0, and version 3.0 and that the vendor will be put out of business by employee turnover if they do not capture architectural knowledge in writing.  The quality vendor has incentive to be thorough and consistent in technical documentation, and to provide the depth and continuity of staffing that makes all elements of the core solution sustainable.

#6 – Maintenance

The question to ask: Will the  system be budgeted and staffed for ongoing maintenance, support and upgrades?

Unless the application solution to be developed is a truly a line-of-business operational crown jewel, the budget and skill-set required for quality ongoing maintenance, upgrades and support are not likely to be well maintained.

Support, maintenance and upgrades are business critical activities for a commercialized technology solution.  A vendor will not have good references and longevity in a marketplace without investing consistently in these areas.

The Success/Failure Criteria Comparison Chart

Here’s another way to digest this information, in the form of a comparison chart:

CRITERIA	INTERNAL SOLUTION	VENDOR SOLUTION
TIMELINE & BUDGET Can the project be completed in a timely manner and close to budget predictions?	Internal project costs are seldom tightly tracked or cost controlled.  Generally everyone keeps their job and gets their full paycheck for the duration, even if the project fails.	Even under poor circumstances, the business has more recourse and cost controls on the vendor solution.  Additionally the commercial solution is simply less likely to fail (to the degree it’s based on real product technology)
FUNCTIONALITY Can the right functionality be achieved and is it competitive with commercial solutions?	Generally, there is little incentive for an internal solution to go beyond minimum requirements. With a single client setting the specs and an internal team developing a one-off solution, there is a significant risk that the results will not go-beyond minimum functionality and will be well below commercial standards.	Commercial technology solutions can be vetted against their longevity, client references, and competing products.  Furthermore, the vendor is incentivized to make the investment that keeps the product technology competitive in the marketplace.
TESTING Can you provide a solution that is  well-tested and bug-free? Will the development team be responsive to bugs and issues encountered by users prior to and during deployment?	The discipline of software testing, particularly if significant user interfaces are involved, is challenging in even the best scenarios.  Doing this for a one-off solution with an informal testing team can be problematic.	With a larger use-case library, and more established development and testing teams, commercial tools are much easier to test (even when heavily customized for client use).  References can be checked and performance standards can be set and enforced.
USER DOCUMENTATION Will the technology be delivered with complete and appropriate end-user documentation and training?	Development of documentation and training material is typically an afterthought for internal projects.  Even when it is accomplished, it’s often not well vetted through repeated use and enhancement.	Commercial products and reputable vendors typically have a much more well-developed documentation discipline.  You can even ask to see the documentation before the vendor is chosen.
SYSTEM DOCUMENTATION Will the system architecture be well-documented such that if persons involved “move on” it can be supported by new resources?	Well produced and thorough technical system documentation is rare with internally developed solutions.  It is often treated as an afterthought and, as it adds cost to a cost center, it is quickly sacrificed to help contain budgets.	Vendor solutions are created on the assumption that there will be a version 2.0, and version 3.0 and that the vendor will be put out of business by employee turnover if they do not capture architectural knowledge in writing.  The incentives are to be thorough and consistent in technical documentation.
MAINTENENCE Will the system be budgeted and staffed for ongoing maintenance, support and upgrades?	Unless the application solution to be developed is a truly a line-of-business operational crown jewel, the budget and skill-set required for quality ongoing maintenance, upgrades and support are not likely to be well maintained.	Support, maintenance and upgrades are business critical activities for a commercialized technology solution.  A vendor will not have good references and longevity in a marketplace without investing consistently in these areas.

 Conclusion

Naturally, there are companies out there with internal technology teams that defy the odds and run efficiently and sustainably.  In my experience, however, there is a large gap between the number of firms that actually do this and the number that only claim to do so.  The problem is that there is not a competitive market place (i.e., a free market) which allows the internal team to be forced to prove it.

In the arena of developing, deploying and supporting application technology, the commercial product and vendor community has all of the advantages and motivations to build and maintain quality productized technology. Ultimately, commercial solutions are consistently held accountable under the glaring light of the commercial marketplace. As a technology buyer, this is the type of accountability you want to invest in.