If we just measure driver for crossover, can we just measure the frequency response and use extract minimum phase? I had read a lot of guides and this is what they all use. Of course at the end when we want to verify the phase response of speaker we can't extract minium phase.
Then you should start reading other guides as soon as possible
As long as the system being measured behaves as a minimum phase system, your statement is true.
For loudspeakers this is usually true as long as the driver performs ideal pistonic movements.
That is, the stiffer the material of the driver, the greater the frequency range in which the driver behaves almost like a minimum-phase system.
Here are a few examples of phase frequency responses of "phase*** versus minimum phase" - ordered from good to bad.
As long as the phase frequency responses are parallel, Minimum Phase can also be used instead of Phase.
*** phase = minimum phase + excess phase
1'' metal tweeter - Phase vs. Minimum Phase
0.75'' silk tweeter - Phase vs. Minimum Phase
4'' Midwoofer - Phase vs. Minimum Phase
8'' woofer - Phase vs. Minimum Phase
As an interim solution if you have " well behaved" drivers and do not push the crossover frequency too much, the method works quite well, but certainly not as a general solution to speaker design.
Update: All measurements shown were performed as dual-channel measurements.