Choosing a population health management (PHM) vendor comes down to five criteria that predict whether the partnership actually works: data integrity and transparency, implementation speed, onshore client support quality, relevant certifications, and how well the platform turns insight into a workflow providers will actually use. Most Accountable Care Organizations (ACOs) build their Request for Proposal (RFP) around feature lists instead — dashboards, measure counts, the roster of EHRs a vendor says it connects to — and discover only after go-live that the feature that mattered most wasn’t on the list at all: whether the vendor’s data can be trusted by the physicians expected to act on it, and whether the team behind it responds when something breaks.
That gap exists because PHM vendor evaluation has historically been treated as a technology purchase, when it’s really a data-trust and operating-partner decision. A NAACOS survey reported in 2022 found more than three-quarters of ACOs work with six or more electronic health records (EHRs) — which means the PHM vendor’s ability to aggregate, reconcile, and validate data across that fragmented environment matters more than any single dashboard feature. This guide walks through the criteria that actually separate a good PHM vendor decision from a costly one, with a checklist ACOs can build directly into an RFP.
Why This Decision Is Harder Than a Feature Checklist
Most PHM platforms look similar in a demo. The differences that matter show up only after a contract is signed: whether risk alerts and care gaps populate in a care manager’s workflow in real time or through a nightly batch job that breaks during an EHR upgrade, whether the platform performs as well across six disparate EHRs as it does in a single-system pilot, and whether the vendor’s team actually understands an ACO’s specific value-based care (VBC) contracts and metrics or is reciting generic use cases.
NAACOS’s guidance on data and analytics is direct on this point: data has to be transparent, accurate, and consistent, and it has to surface only what’s relevant so it becomes a helpful tool for providers rather than an added burden. That’s a data-quality and workflow standard, not a features list — and it’s the standard an RFP should actually be evaluating against.
The Five Criteria That Actually Predict Vendor Success
1. Data Integrity and Transparency
The single most consequential criterion is whether an ACO can trust the numbers a vendor’s platform produces. That means being able to trace a metric back to its source data, understanding exactly how a measure is calculated, and having confidence that data pulled from six or more EHRs has been reconciled and de-duplicated correctly rather than just aggregated and hoped for the best.
The real test is the physician. An ACO’s performance moves only when physicians act on what the platform tells them — closing care gaps, recapturing HCC diagnoses, working the risk-stratified cohorts assigned to their panels. Physicians check that guidance against what they see in their own charts. If the two don’t match — a gap flagged that was already closed, a patient attributed who isn’t theirs, a risk score they can’t reconcile — they stop trusting the tool, they disengage, and the performance results don’t move no matter how good the underlying model is. Data that stands up to physician scrutiny is the first requirement, not a nice-to-have.
That is why black boxes are disqualifying. Ask any finalist to walk through, in specific technical terms, how a single patient’s record moves from claims and clinical feeds into a reported quality score or risk score. A vendor that can’t answer that clearly in an RFP won’t be able to explain a discrepancy to a skeptical physician six months into the contract — or to a payer.
Then ask the harder question: does the ACO have direct access to its own underlying data, or only to the vendor’s rendering of it? This is the dividing line that matters most and gets asked least. A platform that shows you a number but won’t show you the records beneath it is asking for trust it hasn’t earned — and when a payer disputes a quality result or a benchmark calculation, an ACO that can’t get to its own source data is negotiating blind. Full transparency in both directions is what makes confidence in the data possible: nothing hidden, every figure traceable down to the patient record, and the client’s data available to the client in a usable form.
2. Implementation Speed
A PHM contract that takes extended months to a year to reach usable data is an unrecovered loss of visibility into cost and quality performance — and in a downside-risk MSSP arrangement, that’s a year an ACO is managing blind. Ask for a specific, dated implementation timeline tied to concrete milestones (first data feed live, first validated dashboard, first provider-facing scorecard), not a general “several months” estimate, and ask for references specifically about how closely the vendor’s actual implementation matched what was promised.
3. Client Support Quality
Population health analytics is not a “set it and forget it” purchase — it requires an ongoing relationship where the vendor’s team understands an ACO’s specific contracts, benchmarks, and provider network well enough to help interpret what the data means, not just deliver it.
Two ways to make this concrete during evaluation. First, ask each finalist vendor for its Net Promoter Score (NPS) and to break down the specific survey questions behind it (responsiveness, understanding of the client’s business, executive engagement), not just the headline number. Second, ask for client references and use them to learn how the vendor actually works with an ACO day to day: does the team operate as an extension of the ACO’s own staff, invested in helping it succeed, or as a distant third party that delivers a file and moves on? Those two answers tell you more about the next three years than any feature comparison will.
4. Relevant Certifications
Certifications are a proxy for whether a vendor’s data processes have been independently audited rather than self-described. For a PHM vendor also handling quality reporting, that means checking for CMS Qualified Registry status, NCQA HEDIS® compliance, and NCQA Data Aggregator Validation (DAV) certification specifically — each of which requires an external audit of how the vendor’s data is collected, validated, and reported, not just a claim on a website.
5. Fit to Your Actual Goals
The best analytics in the world are worthless if they don’t match how your organization works. Two questions matter more than feature counts.
Does the vendor configure to your goals, or make you configure to its product? Larger platforms tend to ship a fixed model and expect the ACO to adapt around it. That works if your contracts and workflows look like the vendor’s reference customer, and creates friction if they don’t. Ask specifically what gets customized during implementation — measure definitions, benchmark logic, scorecard structure, reporting cadence — and what is fixed.
Where does analytics need to land to change a decision? For some ACOs the answer is alerts embedded in the electronic health record; for others it’s pre-visit reports, provider scorecards, or a quarterly financial review with the analytics team. Decide which of those actually changes behavior in your organization before evaluating vendors on it, rather than accepting a demo’s assumption about where insight belongs.
A related caution: treat artificial intelligence claims as a question, not a credential. Ask what the model does, what it was trained on, how its output is validated, and what happens when it’s wrong. A vendor that answers those clearly is worth listening to. A vendor that uses AI as a category label is telling you about its marketing, not its product.
A Buyer’s Checklist for Your RFP
Build these into your RFP directly, and score vendors on specific answers rather than yes/no checkboxes:
- A walkthrough of how the vendor reconciles and validates data across at least six EHRs and multiple claims sources.
- Written confirmation of what direct access you have to your own underlying data, in what format, and whether that access survives contract termination.
- A description of what happens when a physician challenges a care gap, an attribution, or a risk score — who investigates, how fast, and how the correction gets back into the platform.
- A dated implementation timeline with named milestones, plus two reference clients willing to discuss whether that timeline actually held.
- The vendor’s current NPS along with the specific survey categories behind it — and, separately, any KLAS rating, understanding that the two measure different things.
- Client references you can ask directly whether the vendor’s team functions as a partner or a distant third party.
- Documentation of CMS Qualified Registry, NCQA HEDIS®, and NCQA DAV certifications, with certification dates.
- A specific list of what gets configured to your organization during implementation versus what is fixed product.
- For any AI capability claimed: what the model does, what it was trained on, how output is validated, and what happens when it’s wrong.
- A client reference from an ACO of comparable size and EHR complexity to your own — not the vendor’s flagship account.
Why Net Promoter Score Belongs in Your Evaluation
Net Promoter Score (NPS) — the widely used customer-loyalty metric based on the question “how likely are you to recommend this company to a colleague?” — is one of the few evaluation criteria that reflects the lived experience of an ACO already using the platform, rather than a vendor’s own description of its capabilities. Respondents are scored 0–10 and grouped into promoters (9–10), passives (7–8), and detractors (0–6); a company’s NPS is the percentage of promoters minus the percentage of detractors, which puts the possible range at –100 to 100. That methodology comes from Bain & Company, which created NPS. By common industry convention, scores above 50 are treated as excellent and scores above 80 as rare.
NPS and KLAS measure different things — don’t compare them
This trips up a lot of vendor evaluations. KLAS ratings and NPS are both credibility signals, but they are not the same metric and the numbers are not interchangeable.
KLAS surveys ask, in effect, did you get what you thought you were getting? — measuring whether a product delivered against the expectation set during the sales process, scored on its own performance scale. NPS asks whether a client would put their own reputation behind recommending you, on a −100 to 100 scale.
A KLAS score in the 80s and an NPS in the 80s look similar and mean entirely different things. Treat them as two separate questions worth asking — expectation-fulfillment on one hand, advocacy on the other — and be skeptical of any vendor that presents one as though it were the other, or benchmarks its NPS against a KLAS category average.
What Good Looks Like: Koan Health’s Benchmarks
Koan Health was built on more than three decades of claims analytics experience and has generated over $800 million in Medicare shared savings for its ACO and Clinically Integrated Network (CIN) clients.
Certifications. Koan Health holds CMS Qualified Registry status, NCQA HEDIS® compliance, NCQA Certified DAV Partner status, and HITRUST r2 certification.
Client support. In its 2024 client survey, Koan Health earned a Net Promoter Score of 85 — scoring 100 on account executive responsiveness, 100 on account executive understanding of the client’s business and performance metrics, and 95 on executive leadership investment in client success. Koan Health’s entire team is based onshore in the U.S., removing a common data-security concern for ACOs evaluating vendors that offshore any part of data handling or support.
Data transparency. Clients have access to their own underlying data, not just the platform’s rendering of it — the traceability standard described in criterion one, and the reason Koan Health’s numbers hold up when a physician or a payer asks where a figure came from.
Fit. Koan Health builds to the goals and contract structure an ACO actually has, rather than asking the ACO to adapt to a fixed product model. That is a deliberate choice about who the platform serves: organizations that need a genuine analytics partner without the cost structure of the largest enterprise vendors.
.svg)