A new report from Bowel Cancer UK says lives could be saved by relaxing the rules governing who GPs can refer for urgent bowel cancer tests.
The charity suggests there are also problems with waiting times, and with the quality of tests carried out when doctors have significant worries about bowel cancer.
The report - Diagnosing Bowel Cancer Early: Right Test, Right Time - found a third of patients sent for an endoscopy had seen a GP more than three times before being sent to a specialist.
And almost half of them were found to have cancer, according to the country-wide survey based on 708 responses and carried out in September 2013.
When referring patients suspected of having cancer, current guidelines allow doctors to flag a patient as either urgent or non-urgent when they refer them. But GPs are only allowed to send people for 'urgent' bowel cancer tests if they have symptoms graded as 'high risk' or 'alarm', such as bleeding.
But Bowel Cancer UK says only one in every two people eventually diagnosed with cancer initially goes to a doctor with symptoms that would allow urgent tests.
Deborah Alsina, the charity's chief executive, said it wants the rules for GPs to be "liberalised". She said doctors should be able to send people for cancer tests based on their opinion even if their problems don't tick all the boxes.
Cancer Research UK welcomed the findings but offered a note of caution. Commenting on the report, Cancer Research UK's head of early diagnosis Dr Jodie Moffat said any changes regarding increased testing should be weighed up against the risk of causing unwarranted stress in patients who don't have the disease.
Dr Moffat, added: "Early diagnosis is a vital part of improving survival from cancer. We know that when bowel cancer is diagnosed at the earliest stage, more than 90 per cent of patients survive at least five years, compared to less than seven per cent of those diagnosed at the latest stage.
"We're very keen to explore new and innovative ways to improve early diagnosis of cancer, including developing alternative pathways and looking at referral thresholds. And we know that patients are often keen for symptoms to be investigated even when there is a very low risk of cancer.
"However this does need to be balanced in the context of the wider demands this would place on the system and the unnecessary anxiety it could generate in patients who turn out not to have cancer. Reports such as this one are helpful for further adding to the evidence that this is a critical issue."
As well as referring people for suspected cancer, the government's Improving Outcomes strategy allows GPs to access a test called flexible sigmoidoscopy for patients whose symptoms may not be cancer, but who still require further investigation. But it is unclear how many GPs are making use of these tests in this way.
The report, "Diagnosing Bowel Cancer Early: Right Test, Right Time," is available online: