i would err on the side of caution & smooth it out nonetheless.

those scratches can be "hot spots" under high cylinder pressures and can get really hot (and act like a glow plug).

to make a long story short, it could cause pre-ignition.

as a rule of thumb you want no sharp edges in your combustion chamber.