Results are presented of searches for continuous gravitational waves from 20 accreting millisecond X-ray pulsars with accurately measured spin frequencies and orbital parameters, using data from the third observing run of the Advanced LIGO and Advanced Virgo detectors. The search algorithm uses a hidden Markov model, where the transition probabilities allow the frequency to wander according to an unbiased random walk, while the J-statistic maximum-likelihood matched filter tracks the binary orbital phase. Three narrow sub-bands are searched for each target, centered on harmonics of the measured spin frequency. The search yields 16 candidates, consistent with a false alarm probability of 30% per sub-band and target searched. These candidates, along with one candidate from an additional target-of-opportunity search done for SAX J1808.4−3658, which was in outburst during one month of the observing run, cannot be confidently associated with a known noise source. Additional follow-up does not provide convincing evidence that any are a true astrophysical signal. When all candidates are assumed non-astrophysical, upper limits are set on the maximum wave strain detectable at 95% confidence, h95%0. The strictest constraint is h95%0=4.7×10−26 from IGR J17062−6143. Constraints on the detectable wave strain from each target lead to constraints on neutron star ellipticity and r-mode amplitude, the strictest of which are ϵ95%=3.1×10−7 and α95%=1.8×10−5 respectively. This analysis is the most comprehensive and sensitive search of continuous gravitational waves from accreting millisecond X-ray pulsars to date.