Faced that problem a lot on AoE.
Until I realized it can be cost-effective in the long run to wall in your berry bushes.
And mining camps. And forests.
Pretty much anything.
Byzantines were laughing their ass off at me until they found out a Wonder of the World was behind one of those walls.
