21 December 2014 04:05


News
BMA  | Burns  | CfH  | DH  | Greenhalgh  | Portal
Twitter RSS Newsletter Send to a friend
11

SCR evaluation finds few benefits

17 June 2010   Fiona Barr

The final report of a three year evaluation of the Summary Care Record has found that it has yet to deliver significant benefits.

Researchers from University College London also found few benefits from HealthSpace, the online portal that enables patients to log health information and view their SCR.

Only one in 1,000 patients invited to open an advanced account to access their SCR had done so. Most people using HealthSpace found the functionality limited and perceived few - if any - benefits from it.

Professor Trisha Greenhalgh, lead author of the 234 page report, ‘The Devil’s in the Detail’, said she hoped the government would look carefully at the researchers’ findings before making decisions about the future of the SCR and HealthSpace.

Professor Greenhalgh, who has recently moved to Queen Mary University of London, said: “This research shows that the significant benefits anticipated for these programmes have, by and large, yet to be realised – and that they may be achieved only at high cost and enormous effort.

“When we did find [benefits], they were subtle, hard to articulate and difficult to isolate. It serves to demonstrate the wider dilemma of national databases: that scaling things up doesn’t necessarily make them more efficient or effective.”

The Department of Health said it welcomed the report and would consider its findings. It reiterated the message contained a letter to the BMA from health minister Simon Burns last week, pledging a review of how patients are informed about SCRs and the content of the record.

The three year, one million pound evaluation found a number of ‘wicked problems’ delaying progress with the SCR and HealthSpace.

These include the difficulty of defining a minimal dataset of key data the SCR should include, the task of ensuring that GP data is complete and accurate, the need to gain informed consent from 50m people - many of whom appear to throw information letters away unread - and the technical and operational challenges of uploading data to the SCR.

Where SCRs were in place, the researchers found evidence of improved quality in some consultations and evidence that the SCR was useful for patients unable to communicate or advocate for themselves.

However, they found no direct evidence of safer care and no impact on consultation length or the number of referrals made. The researchers said the impact of the SCR on patient satisfaction was impossible to assess.

On HealthSpace, they found no evidence of improved personalisation of care, no evidence of increased patient empowerment or ability to manage their long term condition, no evidence of improved health literacy and no impact on data quality or reductions in NHS costs.

The researchers said the most striking characteristic of the SCR and HealthSpace programmes was their scale and complexity.

They said successful introduction of the SCR required collaboration between stakeholders from different worlds with different values, priorities and ways of working and that there were still significant barriers to the widespread use and adoption of such records.

However, they also said greatest progress was made when key stakeholders came together to try to understand each other, even when a consensus was not reached.

Dr Laurence Buckman, chairman of the BMA’s GPs Committee, said the BMA did not oppose shared electronic records in principle but that it had long voiced concerns about the way the SCR programme had been implemented.

He added: “It should not have been rolled out ahead of the findings of this evaluation – which uncovers some very serious problems - being published. We are pleased that the programme is now to be reviewed.

"The BMA is very happy to engage with the government to try to find a way forward that has the confidence of both patients and professionals.”

Opinion and analysis: Read more about the SCR evaluation in Fiona Barr's opinion and analysis.

Links: Full report

Summary report

BMJ paper


Related Articles:

9 News: Burns says NHS tech will be reviewed | 11 June 2010
9 News: DH and BMA say as you were on SCR | 5 May 2010
1 News: SCR evaluation costs £1m | 9 April 2010
Last updated: 17 June 2010 09:46

© 2010 E-HEALTH-MEDIA LTD. ALL RIGHTS RESERVED.


Follow

Evaluation should have looked wider than the SCR

Unknown 234 weeks ago

It is a pity that the terms of reference for the evaluation of the SCR did not include detailed comparison with various local initiatives to develop shared records.

Although these projects share many of the 'wicked' problems that Prof. Greenhalgh and colleagues describe, local variants include a number of permutations of GP records, hospital pathology and radiology reports, hospital discharge and clinic letters, and community therapies records. These clinically rich shared records are used to varying degrees by GPs (both in and out of hours), GPs with Special Interests, Hospital Consultants, A&E staff, hospital pharmacists and community nursing and therapy staff.

Some of these shared records are delivered by GP system suppliers, others by specialist companies able to integrate records from a number of different GP system suppliers and hospital sources. A conservative estimate would suggest that there are more than 10 times as many patients who have a locally shared record as have an SCR.

Unfortunately these local projects have typically been undertaken with less generous resources than the National Programme, and have had little opportunity to commission detailed evaluation. If CfH had not ignored the output of the ERDIP projects and been more open-minded about ways to achieve locally as well as nationally shared records, we would have had a much more useful report on what benefits can be delivered by the various different models of sharing records and, given that in most localities more than 90% of emergency admissions and A&E attendances are local patients, the important question of what additional benefit is provided by a national summary record could also have been addressed.


Reply
Flag
 
Like

This comment is:

Email address:

Submit

Follow

Post

 

Expertise:
Sector:
Approved posts:
Likes:
My EHI score:
Reward badges:

Opportunities squandered

Unknown 235 weeks ago

Oh NPfIT, what we could have done if you'd not been invented !

As someone closely involved in trying to get value from SCR at a local level, I see lots of common sense in the SCR report. The key problems have been with delays to technical compliance from GP system suppliers (and apparent failure by CfH to have adequate mechanisms to enforce this), and opposition by vested interests (especially GPs who might just not really want their records shared).

One very visible product for us has been the number of patient consultation letters returned, for those no longer known at that address, but for which the GPs were still getting paid. Some had moved and failed to inform the NHS, but many have ... well, disappeared.

But with key advice that it was not worth an urgent care setting using SCR until at least 60% of the local population to them were likely to have an SCR, the delays around GP system uploads (and an entirely separate CfH decision to include patient phone number in the Spine matching process), we have not yet gone live in all the relevant settings.

Perhaps winding back to the original justification for the overall NPfIT programme, patient safety, the one aspect that Greenhaulgh et al do recognise is that in some of the 40 cases, medication decisions were aided by urgent care having access to SCR.

With no sharing to the NHS of the theoretical SCR benefits that were apparently given to Treasury, this safety argument is the key, that we have shared with patients and staff, and it appears is vindicated in a few cases.

It could have probably been much more economically achieved by building on the previous ERDIP projects in locally joining up records.

Our biggest problem of all has in my view been a lack of empowerment by intervening layers of bureocracy, and an overburdensome implementation of PRINCE2 that did not actually add value.


Reply
Flag
 
Like

This comment is:

Email address:

Submit

Follow

Post

 

Expertise:
Sector:
Approved posts:
Likes:
My EHI score:
Reward badges:

SCR being held to ransom

Unknown 235 weeks ago

It is quite clear that SCR would have been much further ahead and working if some parts of the NHS had supported this for the sake of patients rather than pounds. Lets be clear its all about money not care. When the money is in pocket the dancers will dance.

How about a real life review to see how much the SCR is used.  Look at the Scottish ECR and look at the evaluation of usage and number of accesses.  The figures matched with the populations clearly shows the opportunity SCR gives us.

Has anyone looked at patient safety, the number of mistakes which are made and the number of deaths.  Has anyone looked to opportunities SCR offers regarding violent patiets, end of life care, or how about trying to do something about trying catch child abuse cases. Clearly NO.

there is only one group losing here and it is the patients, say whatever fancy words you like!


Reply
Flag
 
Like

This comment is:

Email address:

Submit

Follow

Post

 

Expertise:
Sector:
Approved posts:
Likes:
My EHI score:
Reward badges:

Right technology - Wrong direction

ROBERTSAND 235 weeks ago

The review of the SCR's slender benefits rings true to me. Well done for highlighting the poor value for money that the taxpayer has derived from this politically motivated white elephant.

What worrys me most is that community electronic records will be damned by association if a knee jerk reaction to the report is adopted by clinicians and politicians.

Top down implementation produces a record that lacks useful detail, ownership and accuracy. Building records locally overcomes this with data that is present because clinical activity is ongoing rather than just in case it might be needed. Ownership is real as the maintainers of the data are responsible to the patients in their communities and depend upon maintaining accurate, secure records for clinical care.

So please - don't throw the baby out with the bathwater!


Reply
Flag
 
Like

This comment is:

Email address:

Submit

Follow

Post

 

ROBERTSAND
Expertise:
0
Sector:
0
Approved posts:
1
Likes:
0
My EHI score:
1
Reward badges:

"GCSE statistics"

Unknown 235 weeks ago

The person who posted a comment 'GCSE statistics' and claimed to be quoting directly from the SCR evaluation report was misquoting.  The sentence attributed to the report ("Of the 237 patient encounters studied in the report, it is understood that less than 40 had a SCR")  is incorrect and not in the report.  The comment also wrongly implies that the overall sample for the study was 237. This is incorrect by three orders of magnitude.

The UCL evaluation studied a quantitative dataset of 416,325 encounters in primary care out-of-hours and walk-in centres in pilot sites where the SCR was said to be in active use. In that dataset, fewer than 4% of encounters involved an access of the patient's SCR. The full report and appendix offers a detailed statistical anlysis of the factors associated with SCR use (and non-use) based on this large dataset. Two key findings were that (a) many patients seen in emegency and unscheudled care did not have a SCR, even in sites where it had been introduced and (b) when a patient's SCR existed, clinicians chose not to view it in 78% of cases.

In order to generate hypotheses on WHY we got these quantitative findings, we studied a further dataset of 237 patient encounters QUALITATIVELY i.e. we sat in on consultations and made detailed notes on what happened and why.  This approach is known as ethnography and is a well-established technique in computer science to consider socio-techncial embedding of technologies. We saw a SCR viewed in fewer than 40 of these cases. In a substantial fraction of the remainder of the cases, the patient MIGHT have had a SCR but the clinician didn't look, hence (as non-participant observers) we could not tell whether it existed or not.  A non-accessed SCR is unlikely to bring benefits.

The full report describes the multiple, complex, interacting influences which explain use and non-use of the SCR in different clinical settings. The full report also documents (paragraph 4.4.3) that local project managers were expected by Connecting for Health to "set up and populate a benefits database" for the SCR and that lack of clearly demonstrable benefits from the SCR was a recurring complaint by these project managers in their regular meetings with CFH.  Our report also documents (paragraph 4.6.4) that an internal inquiry by CFH, described to us as a "draft", raised concerns that front-line staff were not perceiving benefits.

May I suggest to my 'GCSE statistics' critic that it might be helpful to read the report before seeking to denigrate it?

 


Reply
Flag
 
Like

This comment is:

Email address:

Submit

Follow

Post

 

Expertise:
Sector:
Approved posts:
Likes:
My EHI score:
Reward badges:

access links for references (1) and (2)

Unknown 235 weeks ago

(1) Greenhalgh T, Stramer K, Bratan T, Byrne E, Russell J, Mohammad Y et al. Summary Care

Record Early Adopter programme: An independent evaluation by University College London.

 

(1) Greenhalgh T, Stramer K, Bratan T, Byrne E, Russell J, Mohammad Y et al. Summary Care

Record Early Adopter programme: An independent evaluation by University College London.

 

London: University College London; 2008.

 

(2) Byrne E, Kalra D, Stramer K, Greenhalgh T. Data quality evaluation for the Summary Care Record.

 

London: University College London; 2008.

 

(2) Byrne E, Kalra D, Stramer K, Greenhalgh T. Data quality evaluation for the Summary Care Record.

 

London: University College London; 2008.

http://www.haps.bham.ac.uk/publichealth/cfhep/documents/NHS_CFHEP_002_Data_Quality_Supplement_Final_Report_2008.pdf"> color="#0000ff" size="2">http://www.haps.bham.ac.uk/publichealth/cfhep/documents/NHS_CFHEP_002_Data_Quality_Supplement_Final_Report_2008.pdf href="http://www.haps.bham.ac.uk/publichealth/cfhep/documents/NHS_CFHEP_002_Data_Quality_Supplement_Final_Report_2008.pdf"> color="#0000ff" size="2">http://www.haps.bham.ac.uk/publichealth/cfhep/documents/NHS_CFHEP_002_Data_Quality_Supplement_Final_Report_2008.pdf href="http://www.haps.bham.ac.uk/publichealth/cfhep/documents/CFHEP_002_SCRIE_Final_Report_2008.pdf"> color="#0000ff" size="2">http://www.haps.bham.ac.uk/publichealth/cfhep/documents/CFHEP_002_SCRIE_Final_Report_2008.pdfLondon: http://www.haps.bham.ac.uk/publichealth/cfhep/documents/CFHEP_002_SCRIE_Final_Report_2008.pdf"> color="#0000ff" size="2">http://www.haps.bham.ac.uk/publichealth/cfhep/documents/CFHEP_002_SCRIE_Final_Report_2008.pdf


Reply
Flag
 
Like

This comment is:

Email address:

Submit

Follow

Post

 

Expertise:
Sector:
Approved posts:
Likes:
My EHI score:
Reward badges:

semantic interoperability

Unknown 235 weeks ago

The data set used in woefully short of what was required to produce meaningful results,  the author(s) stops just short of acknowledging this, and as such the report is littered with ifs and buts.

In addition the report fails to see the bigger picture that producing a SCR facilitates semantic interoperability between the myriad of software platforms available across the NHS sector.  This has caused vendors to look at standards such as common entities, artefacts, codes and standard APIs consistent across the board. 

 

This shift will offer huge benefits long term for health planning,  data summaries,  patient controlled health record,  inter organisation data transfer, this list is almost endless, but will reduce costs and increase clinical safety.

 

It a big challenge as the NHS is fantastically large organisation, however perseverance will prove fruitful in the long term.

 

Daniel


Reply
Flag
 
Like

This comment is:

Email address:

Submit

Follow

Post

 

Expertise:
Sector:
Approved posts:
Likes:
My EHI score:
Reward badges:

no surprise

Unknown 235 weeks ago

A lot of what is in the NPfIT business cases was rather optimistic (and kept secret from the taxpayer - FOI anybody?) - how else could the justify spending millions on IT when we are struggling to staff our hospitals adequately?


Reply
Flag
 
Like

This comment is:

Email address:

Submit

Follow

Post

 

Expertise:
Sector:
Approved posts:
Likes:
My EHI score:
Reward badges:

GCSE Statistics

Unknown 235 weeks ago

To quote the report summary "Of the 237 patient encounters studied in the report, it is understood that less than 40 had a SCR" which is not really a valid statistical sample from which to identify benefits.  There are now more than 1,658,697 SCRs so let us look at the benefits now!


Reply
Flag
 
Like

This comment is:

Email address:

Submit

Follow

Post

 

Expertise:
Sector:
Approved posts:
Likes:
My EHI score:
Reward badges:

As it says the detail reveals all.

Paul Cundy 235 weeks ago

Buried in the UCL report are the claims the SCR team made to get their business case through the treasury. This is what they claimed for how much the SCR would "speed up" consultations;

  • GP out-of-hours telephone calls, base visits and home visits 15% shorter
  • A&E and walk-in centre encounters 33% shorter
  • attendances at acute medical, surgical and elderly assessment units 50% shorter
  • mental health crisis intervention encounters 60% shorter

Its actually quite difficult to know what to type next! Were they delusional or was it just bare faced lying?

No one with any experience of using electronic patient records would ever have fallen for those claims.

Regards

Paul C

 


Reply
Flag
 
Like

This comment is:

Email address:

Submit

Follow

Post

 

Paul Cundy
Expertise:
Clinician
Sector:
Other
Approved posts:
101
Likes:
0
My EHI score:
101
Reward badges:
 
[1] 2

About us | Advertise | Terms and conditions | Community and comment rules | Privacy policy | Cookie policy | Contact us
Digital Health Intelligence Limited
EHealth Insider is managed and maintained by Digital Health Intelligence © 2014
Registered Office: Southbank House, Black Prince Road, London SE1 7SJ
Registered No. 9257440 | Vat No. 198 3531 71