Would-be homebuyers are worried they'll be left behind