The Royal College of GPs says patients are already refused appointments millions of times a year because of shortage of capacity.
Its figures relate to the number of times patients have to wait for longer than a week for an appointment and come from an analysis of the latest GP patient survey.
This suggests that patients struggled to get appointments some 41.9 million times last year - and that the figure could rise to 46 million this year.
College treasurer Dr Helen Stokes-Lampard said: “The fact that patients in England will be unable to see their GP when they want to on more than 50m occasions in 2015 is a truly shocking indictment of the crisis that is enveloping general practice.
“The profession has been brought to its knees both by a chronic slump in investment and the fact that there are now simply not enough family doctors to go around.
“Whilst some of these patients will try calling the practice another time to get an appointment, this isn’t good enough – many will either ending up in hospital or, worse still, will not seek medical treatment at all."
Some two-thirds of clinical commissioning groups devote less than 10% of their budgets to these services, according to a survey conducted by Labour.
In some places, such as Surrey Heath, the share of the budget shrinks to 6.6%.
Shadow public health minister Luciana Berger obtained the figures using Freedom of Information requests.
In contrast several CCGs in London devote more than 15% of their budgets to mental health services. In West London, the proportion of spending reaches 18%.
Royal College of Psychiatrists' president Professor Sir Simon Wessely said the levels of spending appeared to bear little relation to need.
He said: "We have serious concerns that some of the CCGs with the highest levels of mental health need are spending significantly below the national average.
"Although many CCGs are certainly getting it right, there are those who need to urgently reconsider whether they are giving mental health parity of esteem with physical health, as their patients rightly deserve."
He added: "Over the past two years we have seen a doubling in the number of mental health patients being forced to seek emergency treatment away from their local area - from 1,301 people in 2011-12 to 3,024 in 2013-14, including patients being sent 300 miles from Devon to Yorkshire.
"This is a direct result of not enough money being made available locally for mental health beds."
Ms Berger said: "Mental health should be treated no differently to physical health. People with mental illness shouldn't have to expect different standards of care simply because of where they live."
* The cost of making NHS managers redundant in the last four years has proved double the government's planned total, Labour claimed yesterday.
It said the cost of management redundancy had now reached £1.6 billion as a result of the NHS reforms introduced last year.
St Bartholomew's Hospital will specialise in heart care while University College Hospital will specialise in five kinds of cancer.
NHS England said the proposals marked a "revolution" in care, providing a "brand new way" of delivering highly specialist services in north and east London and west Essex.
It said Barts will become the world's biggest centre of excellence for adults with congenital heart disease. Under the plans it will take over services from the London Chest Hospital and The Heart Hospital.
Under the proposals, other hospitals in the area will continue to provide common heart and cancer treatments - but working in conjunction with the two specialist hospitals.
NHS England regional director Anne Rainsberry said: "This programme is a brand new way of delivering care and a once-in-a-lifetime opportunity to improve services for today's patients and future generations.
"These 'centres of excellence' should also position London as a leader in research into cardiovascular disease and cancer, attracting some of the brightest and best clinicians and academics to work and study in the UK.
"We know that when people need specialist care, they should be treated in centres with the most highly trained and experienced staff and best facilities. However, people also need to receive high quality care close to home too and this new approach will ensure they get that at every stage."
The duty comes into force in hospitals in October together with a second new requirement that senior managers and directors are "fit and proper persons."
Both have been introduced following the public inquiry into the Stafford Hospital scandal.
Now the Care Quality Commission - which will enforce them - has launched a five-week consultation on how they should be put into force.
The duty of candour will require staff and managers to be frank with patients and their families when mistakes are made or when problems occur with treatment.
Senior hospital managers called for improved guidance on how hospitals can be "open and transparent" rather than the introduction of penalties for minor breaches of the duty of candour.
NHS Confederation chief executive Rob Webster said: "It is right that breaches of the fundamental standards be treated with utmost seriousness, and that the CQC can respond promptly.
"It is also important that the CQC respond proportionately and that any prosecutions focus only on the most serious failings in care.”
He added: "We strongly urge the focus now to be on how to create the right conditions to support and train staff to raise concerns, and to make them feel at ease having open and honest conversations with patients, their families and their carers.
"A climate of openness and transparency is one of the ways in which board and individual board members will be aware of systematic examples of poor care or neglect in their increasingly complex organisations."
Commission chief executive Dave Behan said: "Where our inspection teams identify poor care, this guidance will help us to determine whether there is a breach of regulations and if so, what action to take. In some cases, this will mean we will use our powers to prosecute."