If you enter a birthday value into a contact record, Outlook will add it to
the calendar.  I don't think there's any way to stop that.  You can always
delete the event once it gets added.  Unless you perform some other update
on the contact record, Outlook won't keep re-adding it.  If you do modify
the contact record, though, Outlook will re-add it.

Once thing you could do, though, is to assign the "Birthday" category to all
birthdays, then modify the calendar view you're using, creating a filter
which uses the "Categories" field "doesn't contain" "Birthday" condition to
prevent the view from displaying the birthdays.  May thins a new view
(rather than modifying the existing view) and you can then switch between
the standard vew and your view to toggle the display of bithdays.