I am consulting to a company that needs to purchase Medical billing software that will also support payroll given unique business rules. I worked through the requirements with the company and authored a detailed RFP which has gone out to Vendors of Off the Shelf Systems. The client wants to go with custom software if we can't find an adequate OTS solution and for a variety of reasons, I fear there's only a 60/40 chance that an OTS solution will emerge. My question: Does anyone know of good resources/methodologies for evaluating custom software development companies. I could send out a similar RFP to the one above but I fear the companies in question would just say, "yeah, we can do that" and I wouldn't have a good way (outside of asking for references, financial stability, project managment experience...) to choose the few companies to focus on. Alternatively, I've thought about sending out an RFI that would just focus on organizational strengths and experiences then provide each company with my requirements (in the form of written "The system shall...", user stories and diagrams) and ask each company to provide a white paper indicating how they would solve the problem (including communication processes, software development processes... I would then rate them based on who I believed and get them to demo similar software and follow up on references. In this latter approach, I would invite (and pay expenses?) for them to come to us so we could see how they do requirements validation. Any thoughts on alternative approaches or modifications/pitfalls to the approaches I laid out above would be greatly appreciated.
On the custom software development side most clients I work with elect to build teams by selecting individuals from various companies, rather than picking a team from one company. While they lose the potential synergy that a team with a long track record has together, and it often takes longer to build the team, they often find that they can find a higher level of overall talent this way.
I have been involved in choosing vendor teams for non-software projects before, for example a market research team. Our requirements, scope and preferred timeline were included in an RFP. We developed an evaluation criteria matrix to compare the variety of responses that we received, with no emphasis on checking off our requirements (as you indicated they can all say 'yes we can do that'). In the end we had 20 evaluation categories. At a high level we wanted to have the vendor demonstrate:
Based on your situation I think many of the above criteria could fit well into your evaluation. I think going a full RFP is more appropriate; in my experience you don't get enough detail in an RFI and will end up having to issue an RFP afterwards anyway. After we evaluated our responses we short-listed candidates (3) and had them fly in (at their own expense) to give us a presentation that filled in some of the blanks that we felt were missing from their responses, and answer some questions that came up based on the responses.
Interesting statement. I agree many people operate this way, but I am sceptical of the benefit being real. Formed teams with stable systems will outperform new teams with yet to be established systems almost every time.
I've seen both work well and work poorly. I've seen thrown-together teams work well when the project is smaller (less than 10 people, 12 month timeframe). Sometimes the vendors have experienced a lot of churn within their teams so in reality their groups are not stable either.
I agree that having a well-proven system improves performance, regardless of how the team is formed. If the client has a good project methodology this can really even out the learning curve for a thrown-together team.
Thank you for sharing this situation. Evaluating custom software development companies can indeed be tricky, especially when you're dealing with a specialized industry like medical billing and payroll. Your approach of using an RFI followed by a detailed RFP seems solid, as it allows you to assess a company’s methodology and experience before diving into specifics. Here are a few additional suggestions that might help refine your process:
Initial Screening: Before diving into in-depth RFPs, you might consider conducting a round of preliminary interviews or a "soft" RFI that helps you narrow down the list of potential candidates. In these interviews, focus on understanding each company's domain expertise, experience with similar projects, and their approach to solving complex business rules.
Look for Case Studies or Proof of Concepts: When asking companies to provide a white paper, also request detailed case studies of similar projects they've worked on, specifically in the healthcare or finance space. This will give you a sense of how they handle complex requirements and customization.
Focus on Agile Development: Since your client needs a tailored solution, it's crucial that the company has strong experience with agile methodologies. Make sure they can demonstrate how they adapt to changing requirements and ensure regular communication and feedback loops.
Cultural Fit and Collaboration: As part of your evaluation, pay close attention to how companies approach client collaboration. The best partners for a complex project will be those that view themselves as an extension of your team, not just as service providers. Perhaps conduct mock discovery sessions or workshops to see how they engage with your team and gather requirements.
References and Demos: As you mentioned, speaking with references and seeing live demos of their work are essential. When checking references, ask very specific questions about how the company handled challenges, changes, and the delivery process for previous clients in similar industries.
Evaluating Long-Term Viability: Since this is a custom solution, consider the long-term partnership potential. Evaluate the company's financial stability, growth trajectory, and their ongoing support and maintenance processes. This is important to ensure they can continue to support and enhance the software over time.
For a comprehensive review of development companies and their services, you might find this resource helpful: https://www.cleveroad.com/blog/software-development-companies-vancouver/.
Best of luck with your project!
brought to you by enabling practitioners & organizations to achieve their goals using: