It depends on what the defects are. But distilling without immediate mineralization is not a good practice
Yes, to be clearer, by sub-par I wasn't referring to serious contamination so much as smell/taste. In Sydney we have reasonable tap water but it often has too much chlorine for my taste, so I don't usually drink it.
The health issues are more complex, and interesting. Drinking distilled water increases diuresis and discharge of electrolytes from the body, not necessarily beneficial. WHO drinking water recommendations include magnesium minimum of 10 mg/L with 20–30 mg/L optimum; calcium 20 mg/L minimum and 40–80 mg/L optimum and total water hardness (adding magnesium and calcium) of 2–4 mmo/litre, so distilled water is not optimal per those criteria. Distillation will remove fluoride, which may or may not be your health goal. Personal factors may be significant: for example I have good blood serum metrics generally but oddly low serum potassium, drinking distilled water would be detrimental there.
But there are a large number of potential contaminants in drinking water including VOCs, phenols, endocrine disruptors and so on that find their way into the water supply, along with chlorine/bromine disinfection byproducts such as trihalomethanes and haloacetic acids that are detrimental. Reticulated water systems are designed to deliver chlorine residual to ensure effective control of pathogens, so a strategy to remove residuals and contaminants prior to consumption but retain or supplement beneficial minerals etc is logical. More complex than just measuring TDS though.