Auto Upgrade Caused or Exposed CSRF

[williamconley]

Today our system auto-upgraded and we immediately noticed that downloads of PDFs failed. Of course, this makes it rather difficult to download employee pay stubs.

Upon investigation, we found this error:

Code: Select all

code":"EXCEPTION","description":"Invalid referrer, possible CSRF.","record_details":{"total":0,"valid":0,"invalid":0},"user_generic_status_batch_id":false,"details":false}}

Tossed by:
/api/json/api.php?SessionID=4628e12c0aee596db95ab7ec5ddc7911ce401885&Class=APIPayStub&Method=getPayStub&MessageID=C6D72B4C-9695-231B-883E-24182601E623

Apparently this json api ordinarily results in the download of our PDF pay stubs, but after the upgrade it was under the impression there is some cross-site-scripting going on and fails instead. Or perhaps it merely is no longer receiving what it expects and this is a default error.

I don't spend a lot of time in json, so I reverted and completed payroll.

Since full reversion would not tell me if the data was invalid (ie: the generated payroll perhaps failed), I left the DB and just reverted the web folder. I now get a warning about application/db version mismatch, but with the same data I now get my PDF download. So it's not the data, obviously.

Since this was the result of an upgrade (I had a notice about being automatically upgraded to 7.4, I believe, when I started payroll), I also checked the timetrex.log and found:

Code: Select all

DEBUG [L0509] [35526ms]:        <b>[Function]()</b>: Stage2 success!<br>
DEBUG [L0055] [35530ms] Array: <b>SystemSettingFactory::isUniqueName()</b>: Unique Name: auto_upgrade_failed
<pre>string(2) "12" 
</pre>

However, this could be a field name being stored as opposed to an actual error. Nothing else looked like an error. The last files to be copied were all in unit_tests from /tmp/timetrex, but there were plenty before that which were copied to other folders and ended with "success" statements (such as Stage2 success! above).

The final log entries show:

Code: Select all

DEBUG [L1561] [35530ms]:        <b>Factory::StartTransaction()</b>: StartTransaction(): Transaction Count: 0 Trans Off: 0<br>
DEBUG [L1627] [35530ms]:        <b>Factory::Save()</b>: Calling preSave()<br>
DEBUG [L1678] [35530ms]:        <b>Factory::Save()</b>:  Updating...<br>
DEBUG [L1561] [35536ms]:        <b>Factory::StartTransaction()</b>: StartTransaction(): Transaction Count: 1 Trans Off: 1<br>
DEBUG [L1627] [35536ms]:        <b>Factory::Save()</b>: Calling preSave()<br>
DEBUG [L1665] [35536ms]:        <b>Factory::Save()</b>: Insert ID: 45572<br>
DEBUG [L1539] [35536ms]:        <b>Factory::getInsertQuery()</b>: Insert<br>
DEBUG [L1571] [35539ms]:        <b>Factory::CommitTransaction()</b>: CommitTransaction(): Transaction Count: 1 Trans Off: 2<br>
DEBUG [L0095] [35539ms]:        <b>TTLog::addEntry()</b>: LogDetail Disabled... Object ID: 12 Action ID: 20 Table: system_setting Description: System Setting - Name: auto_upgrade_failed Value: 0<br>
DEBUG [L1717] [35539ms]:        <b>Factory::Save()</b>: Calling postSave()<br>
DEBUG [L0147] [35539ms]:        <b>Factory::removeCache()</b>: Attempting to remove cache: all<br>
DEBUG [L0153] [35539ms]:        <b>Factory::removeCache()</b>: Removing cache: all Group Id: system_setting<br>
DEBUG [L0147] [35539ms]:        <b>Factory::removeCache()</b>: Attempting to remove cache: auto_upgrade_failed<br>
DEBUG [L0153] [35539ms]:        <b>Factory::removeCache()</b>: Removing cache: auto_upgrade_failed Group Id: system_setting<br>
DEBUG [L1571] [35539ms]:        <b>Factory::CommitTransaction()</b>: CommitTransaction(): Transaction Count: 1 Trans Off: 1<br>
---------------[ 29-Aug-2014 3:59:17 -0400 (PID: 11374) ]---------------

I did not find any related posts via google or on this site when searching for the json api error.

So, is this an easy fix or at least an easy chase ... or is it time to fresh install and start from this week in a new installation?

[shaunw]

Please upgrade to TimeTrex v7.4.1 and try again.

[williamconley]

Would this be to imply that 7.4 was superceded immediately by 7.4.1? And that this is a known problem with 7.4? Or ... is it more of a "hey! try this and see what happens!"

[shaunw]

v7.4.0 would trigger this error if your timetrex.ini.php hostname line was set incorrectly, unfortunately this turns out to be more common then we expected with On-Site deployments, therefore v7.4.1 was released to disable CSRF checks until they are explicitly enabled in the timetrex.ini.php as to avoid catching people off guard.

[williamconley]

And as it turned out, when I changed the hostname in timetrex.ini.php to match the domain name of the server the problem was resolved (after restoring the code). Does present an issue considering that the server can be accessed through different IPs and domains depending on the network connection of the user.

I wonder if this also affects printing the pay stubs for the users? (or is it only a challenge on the admin interface pdfs? Not sure how they would be different, but possible, I guess.)

Thanks for the assist. I think I'm still going to leave auto-update off. Not entirely happy that such a setting jumped into the system without my knowledge. Awkward.

Question: Is there an "upgrade now" button that will upgrade with the same method as the auto-upgrade? This would allow snapshot of the server, pushing the button, running payroll and rollback if there's a problem. Much better than "oops, I think it upgraded this morning, hope nothing's broken ...".

[shaunw]

Obviously we try our best to avoid breaking things, especially with the auto-upgrade and in fact the auto-upgrade is designed such that it slowly releases new versions to a small percentage of users, then gradually ramps it up as time goes by, so if any issues do arise it affects the least number of people as possible.

Anyways, you can run tools/unattended_upgrade.php to force an auto upgrade when you want.

[williamconley]

Excellent. I'll try that next time. Thanks.