Infinity is a philosophical and mathematical concept. It was either invented or discovered by humans, depending on whether you take the postulational (axiomatic) or platonist view of logic and mathematics. As far as I understand, infinity does not exist in reality outside of logic and mathematics. I am unaware of any natural phenomena known to be either infinitesimally small or infinitely large. It is just that we find it convenient to use mathematics that includes the infinitesimally small and infinitely large, when constructing many mathematical models of reality.
Every finite-sized signal acquisition, conversion, storage, and processing system, be it analog or digital, introduces error into the signal through sampling error, conversion error, processing error and if applicable digitization (quantization) error. The errors include systematic errors which can be reduced by clever redesign choices, and random errors (noise) inherent in the functioning of the components. The size, complexity, effort or money (actual cost, not snake-oil inflation) to reduce random noise seems to be a high-power or exponential function of the decrease in random noise, tending to infinity as the noise approaches irreducible quantum uncertainty levels. Of course, with either digital or analog systems, we do not need to spend money to reduce errors further below the threshold of human audibility.
In audio recording and reproduction, the digital approach that is dominant now includes some errors not present in a purely analog system. These are quantization (ADC), digital processing and DAC errors. Errors accrued during digital processing in recording studios amount to perhaps three bits of amplitude or dynamic range. Fairly high accuracy / low error ADC at high sample rates from microphone (amplified) electrical output to bits at reasonable cost is used by recording studios. Modern delta-sigma DACs produce highly accurate analog output at low cost, with low error in the electrical voltage as compared with the mathematical signal implied by the reconstruction filters.
In a purely analog approach involving vinyl records (or analog magnetic traces on tape for that matter), errors include some not present in a digital approach. These include errors in conversion of the microphone's (amplified) electrical output to the geometry of the grooves in the master mold, errors in the pressing of the vinyl, deterioration of the pressed vinyl with age and use, and errors in conversion of the groove geometry to an electrical signal by a turntable, needle and phono cartridge. Certainly, the geometry of the master or pressed vinyl grooves is analog, and therefore "infinitely" divisible (for all practical purposes we can ignore quantum discreteness and randomness at the molecular and lower levels; noise of the geometric imperfections and dynamics of assemblages of billions of molecules is encountered before that). However, it contains a significant level of error as compared with the (analog) electrical output of the microphone. This is due to the noise of the electro-mechanical cutting and pressing processes. No matter how much you spend on the finest steadiest turntable, needle and cartridge with the least noise, you cannot overcome the error/noise already present in the pressed vinyl. The noise in vinyl storage has been estimated from measurements, and it exceeds that in 16/44.1 digital systems by at least three bits. This is also accepted by most posters in this thread. Add to this the various inconveniences and high cost of vinyl records. None of this need prevent anyone from enjoying playback of vinyl records. Certainly, the analog system, including cutting the master and pressing the distributed vinyl as well as playback, can be made more accurate than 16/44.1 digital processing by reducing the noise in the analog conversion processes mentioned previously. However, this would involve systems of much greater complexity, precision and exorbitant cost. And in turn we can switch to higher bit-rate digital sampling/ADC (also "infinitely divisible"), storage and DAC, which is already available at a far lower cost than would be the equivalent analog system. The bottom line is that the cost/accuracy trade-off for the electrical-to-binary-to-electrical conversions is far better than for the electrical-to-vinyl-to-electrical conversions, even though the latter conversion stays in the analog ("infinitely divisible") domain all the way.
The same thing has proved true in the video world when considering optical-to-binary-to-optical versus optical-to-exposed-film-to-optical conversions. You could use the best film stock available to make your photo or movie. If your vision was miraculously capable of discerning infinitesimally small details, you could peer all you wanted at the analog image thrown by your film projector on the white screen and compare it with the original (analog) scene observed directly with the naked eye, and there would be tiny details in the original scene that would not be present in the projected image, and vice-versa. Even though they are both analog, with no digital representation intervening between them. In the digital system, the TV acts as the DAC and amp.