Hey Ankit,
there are a couple of things that could be causing this delay

1: Querying Data sources on start up, when you create data connections in
InfoPath by default they will attempt to execute when the query when the form
loads. If the data from theses connections is not required till latter then
query the data when its required not on load.

2: Mapping fields to events in managed code, if you have  lot of controls on
your form that all have events linked up to them then infopath has to map
each of these events to controls via the data in the Manifest. this is very
slow, to get around this you can create a global event that uses a switch
statement to forward processing on to the correct method.

3:Excessive processing, this is an extension of number 2 basicaly if your
form is running a lot of processing such as validations etc when the form is
loading then this will slow it down as well.

There was a really good presso on this at Teched 2005 I'm still trying to
track down a link to it when I do I'll post back but until then I hop this
helps

Regards
Ben