Quick and easy solution to this is:

Save your worksheet in CSV format - import that CSV file.

We have seen this both "work" and "fail" with an Excel column being
formatted as text. We know "why" it happens(or fails - whatever your
perspective is on the matter)  but not sure exactly what the triggers are
given the different versions of MDAC/Jet, Excel, Outlook (and any
combination of thereof).. For us, it became a waste of effort (effort being
loosely translated to equate to money) to try and figure it out so we
created a generic solution within our import products that eliminates this
issue. If the numeric phone # issue is your only problem - then we've found
that the above suggestion cures the issue 99.999% of the time using the
standard Outlook import wizard.

As an aside for othes reading this - US Zip codes in Excel worksheets can
also exhibit this behaviour.

Karl