Other people seem to have had the reverse problem - wanting it in the
Primary Header and it ending up in the First Page Header and it seems to be
caused by  the image being inserted as an InlineShape.

A couple of fellow MVP's collaborated on the following code as a workaround:

Sub ConvertInlineShapeToShape()
Dim rngShape As Range
Dim rngNewShape As Range
Dim oInlineShape As InlineShape
Dim oShape As Shape

With ActiveDocument.Sections.First
 'get the inlineshape from the primary header
 Set oInlineShape = _
   .Headers(wdHeaderFooterPrimary).Range.InlineShapes(1)
 'Temporarily move the inline shape to the first page header
 Set rngShape = _
   .Headers(wdHeaderFooterFirstPage).Range
 rngShape.Collapse
 rngShape.FormattedText = oInlineShape.Range.FormattedText
 'delete the inline shape from the primary header
 oInlineShape.Delete
 'convert the moved inline shape to a shape
 Set oInlineShape = _
   .Headers(wdHeaderFooterFirstPage).Range.InlineShapes(1)
 Set oShape = _
   oInlineShape.ConvertToShape()
 'Copy the converted shape back to the primary header
 Set rngShape = _
   oShape.Anchor
 Set rngNewShape = _
   .Headers(wdHeaderFooterPrimary).Range
 rngNewShape.Collapse
 rngNewShape.FormattedText = rngShape.FormattedText
 'delete the temporary shape from the first page header
 oShape.Delete
End With

End Sub

Maybe however you can just change your code to put it in the PrimaryHeader
instead of the FirstPageHeader and you may then get it where you want it.