My reply to a similar question last year:

<<
As far as I can tell, this will occur if you havemore than 10,000 records in
your data source and you are trying to select individual records in the
MailMerge Recipients Dialog Box, which can only contain 10,000. 9999 records
and you seem to be OK. Of course, some other factor may be at work as well.

The 10,000 limit is AFAIK undocumented by Microsoft. Actually, people have
mentioned similar problems before and I, for one, have never connected this
particular problem to that particular limit. But if it is the 10,000 limit,
you could try submitting a support incident, but I wouldn't hold your breath
for a solution. Personally, I would have thought that a limit that at least
accommodated the maximum number of rows in an Excel 2003 sheet would help a
lot of people and would not cause problems for present-day PCs.

The only other suggestions I can make are
a. apply query conditions /before/ making your selections, but there can be
problems in that area as well.
b. consider the possibility that when you reachthe 10,000 limit, you're in
the territory where MS is pushing you to do things in a different way. Sad
if that's the case but perhaps a nasty reality.