I once had the same issue and it cost me over 2 hours to solved this question. And I finally get iPhone 4 out of recovery mode loop.
If you're having the same problem, connect your iPhone to your computer. It should open up iTunes and ask you to restore the iPhone in order to take it out of recovery mode. Trust me, I was panicking and was afraid to lose all of my data, but don't worry. To fix it, you do need to restore it, but you need to make sure you have all of your data backed up on your computer! DO not restore it if you don't have your data backed up, this will erase your data completely on your phone. After you've backed it up, you can restore your phone, follow the steps, and wait for your phone to restore to the factory settings. Do not panic like I did, just relax, disconnect your phone from your computer safely, reconnect your iPhone to it again, and iTunes should come up. When it tells you, "Welcome to Your New iPhone", you can choose the option to restore it from your last backup, and let it take time for all of your data to reload into your phone. Remember, be patient with your phone and all will be well. Hope this helps you.