I am consulting to a company that needs to purchase Medical billing software that will also support payroll given unique business rules. I worked through the requirements with the company and authored a detailed RFP which has gone out to Vendors of Off the Shelf Systems. The client wants to go with custom software if we can't find an adequate OTS solution and for a variety of reasons, I fear there's only a 60/40 chance that an OTS solution will emerge. My question: Does anyone know of good resources/methodologies for evaluating custom software development companies. I could send out a similar RFP to the one above but I fear the companies in question would just say, "yeah, we can do that" and I wouldn't have a good way (outside of asking for references, financial stability, project managment experience...) to choose the few companies to focus on. Alternatively, I've thought about sending out an RFI that would just focus on organizational strengths and experiences then provide each company with my requirements (in the form of written "The system shall...", user stories and diagrams) and ask each company to provide a white paper indicating how they would solve the problem (including communication processes, software development processes... I would then rate them based on who I believed and get them to demo similar software and follow up on references. In this latter approach, I would invite (and pay expenses?) for them to come to us so we could see how they do requirements validation. Any thoughts on alternative approaches or modifications/pitfalls to the approaches I laid out above would be greatly appreciated. 

 On the custom software development side most clients I work with elect to build teams by selecting individuals from various companies, rather than picking a team from one company. While they lose the potential synergy that a team with a long track record has together, and it often takes longer to build the team, they often find that they can find a higher level of overall talent this way.

I have been involved in choosing vendor teams for non-software projects before, for example a market research team. Our requirements, scope and preferred timeline were included in an RFP. We developed an evaluation criteria matrix to compare the variety of responses that we received, with no emphasis on checking off our requirements (as you indicated they can all say 'yes we can do that'). In the end we had 20 evaluation categories. At a high level we wanted to have the vendor demonstrate:

• Experience in our industry, not just in doing the work in other industries
• The key resources on the team had relevant experience and were going to be appropriately dedicated to our project (as opposed to working on multiple projects and not giving our project sufficient attention)
• The vendor could articulate their understanding of our objectives and needs through their response. If people just regurgitated what we said we marked them low, if they asked insightful questions, described methodologies that would work well in our situation, etc. they scored much higher.
• They understood the scale and scope of work by developing a work breakdown structure.
• The deliverables proposed were appropriate given the type of work we were looking for them to do.
• They identified risks based on our RFP and described how they can mitigate those risks (we didn't explicitly ask for this in the RFP but gave high marks to those who identified risks on their own).
• Previous references who were either in our industry or where the work performed was very similar, and who we could contact to see how well they had done.
• That they do quality work by evaluating the quality of the response. Spelling, grammar, visual style, etc. of the response were evaluated. 

Based on your situation I think many of the above criteria could fit well into your evaluation. I think going a full RFP is more appropriate; in my experience you don't get enough detail in an RFI and will end up having to issue an RFP afterwards anyway. After we evaluated our responses we short-listed candidates (3) and had them fly in (at their own expense) to give us a presentation that filled in some of the blanks that we felt were missing from their responses, and answer some questions that came up based on the responses.

I've seen  both work well and work poorly. I've seen thrown-together teams work well when the project is smaller (less than 10 people, 12 month timeframe). Sometimes the vendors have experienced a lot of churn within their teams so in reality their groups are not stable either.

I agree that having a well-proven system improves performance, regardless of how the team is formed. If the client has a good project methodology this can really even out the learning curve for a thrown-together team.