Recruitment levels are falling well below targets - while growing numbers of doctors are retiring or leaving to have families, according to the report for Health Education England.
GP leaders welcomed the findings, which they said backed their concerns about the growing pressures on primary care.
The report says planners may have to take action to cut specialty training posts to ensure medical graduates undertake GP training.
The report, Securing the Future GP Workforce, says it is policy to increase GP training numbers to 3,250 a year - but for the last four years it has been stuck at 2,700 a year.
It says GPs were involved in 90% of NHS contacts in 2008 - some 309 million consultations.
The report sets out the challenge of persuading 50% of medical students to train as GPs.
It finds that no more than 30% of medical graduates have ever indicated general practice to be their first career choice.
"This suggests that if General Practice could be promoted more effectively as a career at school, throughout medical school and into foundation training, career preferences and recruitment levels might be improved," the report says.
It warns that problems are being aggravated by growing numbers of doctors who do not complete their training on time - some 15% - and the number of part-time trainees, also 15%, who occupy full-time training slots.
The report adds: "A reduction in specialty training numbers would be consistent with Government policy to shift care away from hospitals, with an accompanying shift of education investment. This will require an extremely sensitive and co-ordinated approach, within appropriate timescales.
"The scale of the service reconfiguration needed cannot be under-estimated but we believe it is necessary to ensure an overall NHS medical workforce with the right numbers in the right places."
British Medical Association GP committee deputy chair Dr Richard Vautrey said the report had been completed in March and it had taken "persistent pressure" to get it published.
He said: "There is no longer any time to waste and the government needs to implement the findings of this report in full and begin a programme of sustained, long term investment in the GP workforce as the BMA has called for in the Your GP Cares campaign."
And some of these costs may be offset by financial savings from improvements in care, according to the National Institute for Health and Care Excellence.
It published its analysis after reports have suggested that hospitals have been pushing their finances to the limit to increase nursing numbers.
NICE said that between 2013 and 2015 hospitals have spent some £1.2 billion in updating care quality following the Francis report into Mid-Staffordshire.
It said that in one year nurse numbers increased by 3% and the number of healthcare assistants by 7%.
NICE says its recommendations for safe staffing are likely to cost another £207 million.
But this will be offset by factors such as reduced numbers of pressure ulcers, reduced risk of infections, fewer legal actions and potential reductions in the time patients spend in hospital.
Professor Gillian Leng, of NICE, said: “Safe care is a good investment which has the potential to significantly reduce costs to the NHS.”
Sir Bruce said the move is his "number one priority."
He said the aim was that patients should get the same services at weekend as at week-day.
Speaking at an event in London, he claimed the solutions to make it happen were "beginning to emerge."
Some 13 areas, including Chesterfield, Derbyshire, are already making moves towards seven-day working in an early adopter programme.
Sir Bruce said: “We have got to stop talking about ‘seven day working’, where the emphasis is on the people delivering the service. We have to talk about ‘seven day services’ and focus on the people receiving the services.
“This is about how and not about why."
He added: “We know in the work we have done already that we are able to find the solutions for providing services at weekends. This is the most exciting challenge we have faced in the last decade, and if we get it right it will make us a world leader.”
Prime Minister David Cameron announced a series of measures at a summit in London yesterday.
These include laws to make parents responsible if girls are injured together with some border checks on children travelling to high risk countries.
Some £1.4 million is to be pumped into a prevention programme in the NHS.
This will include improved training opportunities and improved data collection.
There will also be new guidance on health professionals' responsibilities for safeguarding children.
Earlier reports had suggested that training might become mandatory for clinicians.
Public health minister Jane Ellison said: "I am immensely proud of the work the government is doing to eradicate this dreadful practice. This will enhance the safeguarding responsibility of the NHS around female genital mutilation to care, protect and prevent."
Dr Tony Calland, chair of the British Medical Association's ethics committee, welcomed the proposals.
He said: “Female Genital Mutilation is a form of child abuse that we need to do everything we can to stop.
“Any training that helps improve the identification and prevention of female genital mutilation is to be welcomed. Public sector workers in particular have a key role to play in battling this practice."