The physics is that voltage across a resistor fluctuates because the electrons have random motion unless they are cooled to absolute zero. This is characterized by squaring the voltage that you measure and taking the average of the squared voltage. (If you don't square it, it is positive as often as it is negative, so you get zero average voltage but not zero average voltage squared.)
The formula states that <V^2>=4kTRB, where <> means average, the square of V is V^2, the Boltzmann constant k=1.38066 x 10^(-26) joules per kelvin degree, T is the absolute temperature in kelvin, R is the resistance in ohms , and B is the bandwidth in hertz of the measurement. If you use T=296 kelvin which is 73.13 degrees Fahrenheit, <V^2>=[1.635x10^(-20)]RB.
To get something in dB requires a reference level, and there are many different conventions. One common level is 1 mW (a thousandth of a watt). To compare apples to apples, we need to convert the Nyquist formula to power instead of voltage, which fortunately is not too hard, as P=<V^2>/R. Thus, combining the last two formulas gives P=[1.635x10^(-20)]B. Calling the reference power Pref, the noise power in dB is
10log(P/Pref). A common electronics reference power is Pref=1 milliwatt=10^-3 watts, so the formula becomes 10log{[1.635x10^(-17)]B}. You can see that this does not depend on the frequency, but on the bandwidth or the range of frequencies. If the bandwidth B=1 kHz, the formula says 10log{[1.635x10^(-14]}=-138dB. This is the total noise contained in a one kilohertz range of frequencies, for example, between 2 kHz and 3kHz, or between 10 kHz and 11 kHz.
Notice that the larger the bandwidth, the larger the noise. If, for example, we want the total noise from 0 to 20 kHz, the answer is -124 dB, and this is relative to a 1 milliwatt signal.
I hope that someone checks the math, and also I hope that someone who thinks about acoustics instead of electronics will correct any mistakes I've made, up to and including being completely off the rails here!