right click in the toolbar area and select Visual Basic from the list of
toolbars available.  On that toolbar select the icon with the triangle,
pencil, and ruler.

Now you should be able to select the button and delete it.  Sometimes you
need to Ctrl-click to select

Then click on the triangle, pencil, ruler icon again