With Outlook closed, copy the PST to the portable drive.  Restart Outlook.
It should complain that it can't find your folders and give you a browse
dialogue.  Browse to the PST on the portable drive, select it, and click OK.
Outlook should open and appear as before, but referencing the PST on the
portable drive.

On the other PC, rename your PST, insert the portable, and start Outlook.
When it complains it can't find your folders, again browse to the PST on the
portable.  Outlook should open and look much like the other, barring any
view customizations.

Never remove the drive while Outlook is open.

Also keep in mind that any rules you may have won't easily transport.  If
you intend to use them, you'll have to update them every time you move the
PST between machines.