Right. It's positive feedback makes an oscillator and it's what happens when you stick a microphone in front of a speaker and you get that feedback squeal. (I once built a preamp that turned-out to be an RF oscillator and I fried one channel in my power amp. There was an "unexpected" phase shift that turned negative feedback into positive feedback at very-high frequencies.)
Negative feedback (CORRECTIVE feedback) is an awesome thing!!! No actual engineer would disagree. It's what allows you to drive straight down the road as you make corrections with the steering wheel. It how airplanes find the airport and how ships find the port. Positive feedback would be turning to the right when you start drifting to the right... That would be a bad thing!
The amp, cabinet, and speaker are "part of the instrument". They are not supposed to be "high fidelity", they are supposed to alter the sound. Usually when a guitar is recorded direct (without an amp, speaker, or cabinet} a "sim" (a simulator effect) is used to simulate the effect of using an amp because the signal directly out of the guitar doesn't "sound right". Every guitar player has their favorite amp and, they often use all kinds of effect pedals, etc.
A high-fidelity amp isn't supposed to alter the sound. It's supposed to accurately reproduce the recording. You wouldn't want to use a guitar amp (or effect pedal) as your hi-fi/stereo amp.
If you like extra distortion from vinyl, or it you like the "warm crackle" of vinyl, or if you like to boost the bass, etc., that's OK but it's not "high fidelity" reproduction.