I needed to change the time out for the IFD / ADFS to 10 hours (not as secure as I'd like it!) in CRM 2011.
On the ADFS farm in PowerShell....
Get-ADFSRelyingPartyTrust -Name "externalname.com"
which listed TokenLifeTime: 480....so I changed it to....
Set-ADFSRelyingPartyTrust -targetname"externalname.com" -TokenLifetime 600
