The short answer is that it's a lot safer than not cutting it off.
Some moulds make and release poisons, called mycotoxins, into the food that could, over time, make you very sick. Why they do it is not especially well understood but that doesn't make it any safer.
Some mouldy foods should simply be discarded (ideally, to compost). For others, though, you can salvage and use the unaffected parts without exposing yourself to a health risk. That's good if your mouldy food is an expensive, vintage cheddar cheese!
The life of moulds
Moulds are fungi. They're related to mushrooms, and the yeasts we use to make bread, or convert sugars to alcohol. They are heterotrophs, meaning they can't make their own food (unlike plants). Instead, they degrade complex organic molecules in their environment into smaller molecules they can absorb to meet their energy and nutrient needs.
In nature, mould's ability to break down detritus (waste) ensures that dead matter doesn't accumulate. It also enables the release of minerals that are chemically tied up in detritus to the plants that need them for their primary production.
Moulds are single-celled organisms and, individually, are microscopic. When water and nutrients are available (such as in semi-perishable foods) they grow in number: to procreate, mould cells simply make copies of all essential cell components, and then divide into two new (genetically identical) "daughter" cells.
When moulds divide the two cells stay connected and when they divide again and again, they form a long chain of cells, called a hypha. The hyphae can branch and collectively form a complex matrix called a mycelium that, when big enough, can be seen with an unaided eye. This is the furry growth we can see, for example, on crumpets, berries, jam, tomato paste, cheese, and so on.
The growing tips of the hyphae release enzymes into the environment to degrade complex organic molecules into usable nutrients. The tips of the hyphae also release the mycotoxins which are probably released to ward off competitors.
So, wherever the mycelia go in search of nutrients, toxins may also be found. The extent of spread of the mycelium is not always visible, however, and herein lies the problem.
What to do?
Many moulds can grow on, and spoil, our foods. Among those we are likely to encounter on foods in our homes are Penicillium ("cousins" of those used to make antibiotics, or to ripen some cheeses), Aspergillus, and on fruits, Botrytis.
You're unlikely to experience any immediate symptoms from ingestion of mycotoxin-contaminated foods. Ongoing exposure increases the chances of a range of diseases including include kidney, liver and immune system damage, increased risk of a range of cancers and neurological symptoms; though these worst-case scenarios are rare.
Not all moulds on foods will produce mycotoxins, or produce them at harmful levels, but without a microscope and laboratory its hard to distinguish the dangerous and harmless ones. Given the risk to your health its best to take a very cautious approach to visible mould growth on any food with some exceptions.
A good rule of thumb to judge whether a mouldy food can be "saved" is its moisture content or firmness. Foods with a high moisture content such as cooked casseroles, soft fruit and vegetables, pastes/sauces and soft cheeses can have invisible hyphae growing below the surface and producing mycotoxins.
The same is true for porous foods such as bread and cakes where the hyphae can penetrate. All of these foods should be discarded if you see mould on the surface.
Conversely a cheddar, or a salami, or carrot, that have dense structure are less likely to have extensive hyphal growth away from the visible mycelium. In these cases the mycelium can be cut away (to a centimetre or two depth) and the remaining food consumed with little risk.
The United States' Department of Agriculture's website is a good source of advice for dealing with mould contamination on a wide variety of foods.