Timetrex get punches from Access Control memmory limit

Discussion for TimeTrex open source community developers.
Locked
dritan
Posts: 30
Joined: Sat Feb 13, 2010 4:29 am

Timetrex get punches from Access Control memmory limit

Post by dritan »

HI
We have linked timetrex with another software access control that get punches from one or more doors.
The software in testing environment was going Ok. But when we installed in a real world, it get confused.
So, the users are punching too munch comparing to a time attendance system, and I have done a schedule, that runs the importing of new punches every 10 minutes.
The problem is that if this interval increases, example to 1 hour, it get stuck and when I retry with the importing operation it dubblicate the punches in timetrex.
This is the error that I receive when there are too much information:

C:\TimeTrex>php\php C:\TimeTrex\timetrex\maint\SyncPunch_Korrik.php
PHP Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to all
ocate 3962540 bytes) in C:\TimeTrex\timetrex\classes\modules\core\Profiler.class
.php on line 91

C:\TimeTrex>php\php C:\TimeTrex\timetrex\maint\SyncPunch_Korrik.php
PHP Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to all
ocate 1048637 bytes) in C:\TimeTrex\timetrex\classes\modules\core\Debug.class.ph
p on line 226


Where can I increase this memmory limits or what should I do to avvoid having dubblicate values?
Thanks a lot
Dritan .
shaunw
Posts: 7839
Joined: Tue Sep 19, 2006 2:22 pm

Re: Timetrex get punches from Access Control memmory limit

Post by shaunw »

The memory limits are controlled by PHP itself using the php.ini file.
dritan
Posts: 30
Joined: Sat Feb 13, 2010 4:29 am

Re: Timetrex get punches from Access Control memmory limit

Post by dritan »

Dear Shawn,

Thanks for your help. You are very important here!

We have made a kind of job to import punches to Timetrex from an access control software.
I solved this problem as you described, so I have changed the line at php\php.ini :
memory_limit = 1528M ; Maximum amount of memory a script may consume (128MB default )

Now it's working good, but the problem that I see is that the postgree database is growing a lot (so for a year it's backups are more than 1 GB). I know that you are not supporting older versions (3.05) but, in case I upgrade to the latest one, can you make a kind of "archive" past years (so to remove older punches that are not needed).

Growing so much the database will be difficult to administer and heavy for the computers. My Opinion is that you should think for a solution to shrink the database, or clean from older entries.

Thanks and best regards!

shaunw wrote:The memory limits are controlled by PHP itself using the php.ini file.
shaunw
Posts: 7839
Joined: Tue Sep 19, 2006 2:22 pm

Re: Timetrex get punches from Access Control memmory limit

Post by shaunw »

Newer versions of TimeTrex do clean the database each week to help reduce its size, but a 1GB database should not be a problem at all. We have customers with 100+GB databases and performance is not an issue whatsoever.
dritan
Posts: 30
Joined: Sat Feb 13, 2010 4:29 am

Re: Timetrex get punches from Access Control memmory limit

Post by dritan »

I was thinking in this case to upgrade from 3.0.5 version to the latest version in your site, 5.0.4 version.
I followed the procedure at README.TXT/Upgrade, but in the 3. Database Configuration it was upgrading the database and it it hangs at 83% (even if I let it hours, I see that is not any activity. I tried to make a refresh page, it starts from 69%, goes directly to 83 and again stops there).

Can you help me for this upgrade issue? should I upgrade first to intermediate versions and after to 5.0.4. ?

Thanks
shaunw wrote:Newer versions of TimeTrex do clean the database each week to help reduce its size, but a 1GB database should not be a problem at all. We have customers with 100+GB databases and performance is not an issue whatsoever.
shaunw
Posts: 7839
Joined: Tue Sep 19, 2006 2:22 pm

Re: Timetrex get punches from Access Control memmory limit

Post by shaunw »

v3.0.5 is quite old and hasn't been supported for over a year now, so upgrading to v5.0.4 is not technically even supported or tested anymore. In the future you will need to upgrade more often to avoid such issues.

Regardless, please see this topic for more information on debugging this issue:
http://forums.timetrex.com/viewtopic.ph ... hema#p8170
dritan
Posts: 30
Joined: Sat Feb 13, 2010 4:29 am

Re: Timetrex get punches from Access Control memmory limit

Post by dritan »

Which may be the solution? I had even 3.1.1 version, and 3.4 version that i thought to upgrade step by step, but even for those it hangs at a different point (example 53%).

Another problem for us that I hope it can be solved is that I got this punches from an access control software. I import with import tools at Timetrex. The problem is that the algorithm of Timetrex says: If an consecutive IN (Or Out) punch exist, it throw away waiting for the Out (Or IN punch). So in this case if a person punches IN at 8'o clock and IN again at 10, it will store the first one, and so It seems to be at work, instead he is cheating the system in this way.

Please help me with this 2 problems. I have seen the logs (activating Debug), but it doesn't shows any useful information.

Thanks Shawn for helping us!
Attachments
timetrex buffer.txt
(4.38 KiB) Downloaded 750 times
Copy (2) of timetrex.log
(16.87 KiB) Downloaded 749 times
shaunw
Posts: 7839
Joined: Tue Sep 19, 2006 2:22 pm

Re: Timetrex get punches from Access Control memmory limit

Post by shaunw »

Unfortunately the debug logs you sent don't contain any useful information. Please follow instructions at the link I sent you, but you need to read the entire topic with all the posts as that person did the exact same thing that you did at first.
dritan
Posts: 30
Joined: Sat Feb 13, 2010 4:29 am

Re: Timetrex get punches from Access Control memmory limit

Post by dritan »

Ok, Shawn, I'll do it again and let's see what will be the result.

What about the second question (importing punches from timetrex tools?)
shaunw wrote:Unfortunately the debug logs you sent don't contain any useful information. Please follow instructions at the link I sent you, but you need to read the entire topic with all the posts as that person did the exact same thing that you did at first.
shaunw
Posts: 7839
Joined: Tue Sep 19, 2006 2:22 pm

Re: Timetrex get punches from Access Control memmory limit

Post by shaunw »

Lets focus on one issue at a time please.
dritan
Posts: 30
Joined: Sat Feb 13, 2010 4:29 am

Re: Timetrex get punches from Access Control memmory limit

Post by dritan »

I have tried to get the log, but it don't show, so I have copied the source of the frame. (attached)
Attachments
timetrex.pdf
(603.79 KiB) Downloaded 930 times
shaunw
Posts: 7839
Joined: Tue Sep 19, 2006 2:22 pm

Re: Timetrex get punches from Access Control memmory limit

Post by shaunw »

Okay, so the issue appears to be that your database is partially corrupted and TimeTrex is not able to create the indexes that it needs during the upgrade process. More specifically there are duplicate rows with the same IDs in the wage_group table. If you are familiar with manually modifying the database directly you could probably find these and remove them, then try the upgrade process again.
dritan
Posts: 30
Joined: Sat Feb 13, 2010 4:29 am

Re: Timetrex get punches from Access Control memmory limit

Post by dritan »

shaunw wrote:Okay, so the issue appears to be that your database is partially corrupted and TimeTrex is not able to create the indexes that it needs during the upgrade process. More specifically there are duplicate rows with the same IDs in the wage_group table. If you are familiar with manually modifying the database directly you could probably find these and remove them, then try the upgrade process again.
Thanks Shaunw... you are the one. I will see what can I do and I will post the results after.

Regards!
dritan
Posts: 30
Joined: Sat Feb 13, 2010 4:29 am

Re: Timetrex get punches from Access Control memmory limit

Post by dritan »

Thanks to your advice, The upgrade database works, after several retries because at this database were different tables containing duplicated values. (the tables doesn't have primary key and was so difficult).

Anyway at the end, after trying to logon with admin credentials, it gives this msg:
TimeTrex is unable to connect to its database, please make sure that the database service on your own local TimeTrex server has been started and is running. If you are unsure, try rebooting your server. (I am working to solve that!)

Now my concern regards the second problem. So using import Tools from timetrex, we can import punches from other databases.

The problem is when 2 or more consecutive punches are the same kind (example 2 consecutive IN punches).
Timetrex algorithm as it is record the first one and drops the others. This is a problem (bug) since for example the employes can cheat the time attendance system punching IN at 8.00 and go to have a coffee and punch again IN at 10.00.
In order to be correct, timetrex should update (replace) the corresponding similar punch before with the latest one.

Can this be possible?

Thanks again and best regards!
shaunw
Posts: 7839
Joined: Tue Sep 19, 2006 2:22 pm

Re: Timetrex get punches from Access Control memmory limit

Post by shaunw »

dritan wrote: Anyway at the end, after trying to logon with admin credentials, it gives this msg:
TimeTrex is unable to connect to its database, please make sure that the database service on your own local TimeTrex server has been started and is running. If you are unsure, try rebooting your server. (I am working to solve that!)
Try clearing your TimeTrex cache directory to see if that helps.

Now my concern regards the second problem. So using import Tools from timetrex, we can import punches from other databases.

The problem is when 2 or more consecutive punches are the same kind (example 2 consecutive IN punches).
Timetrex algorithm as it is record the first one and drops the others. This is a problem (bug) since for example the employes can cheat the time attendance system punching IN at 8.00 and go to have a coffee and punch again IN at 10.00.
In order to be correct, timetrex should update (replace) the corresponding similar punch before with the latest one.

Can this be possible?
The only way around this would require fairly substantial custom development. This is just one of many reasons why we don't recommend using access control systems for tracking time and attendance, because there are hundreds of similar problems that will never have good solutions.
dritan
Posts: 30
Joined: Sat Feb 13, 2010 4:29 am

Re: Timetrex get punches from Access Control memmory limit

Post by dritan »

It seems the problem is not the cache. I have cleaned it, but I see that in timetrex.log it gives an error not finding a column in users table.

timetrex.log atached!
shaunw wrote: Try clearing your TimeTrex cache directory to see if that helps.
Attachments
1 timetrex.log
(21.78 KiB) Downloaded 777 times
shaunw
Posts: 7839
Joined: Tue Sep 19, 2006 2:22 pm

Re: Timetrex get punches from Access Control memmory limit

Post by shaunw »

It would seem that the database upgrade process didn't finish. Have you tried running the installer once more to confirm that everything is complete?
dritan
Posts: 30
Joined: Sat Feb 13, 2010 4:29 am

Re: Timetrex get punches from Access Control memmory limit

Post by dritan »

It was finished, upgrading the DB 100%, and after it continued with other steps. Anyway I will try it again now and post the results!

Thanks!
dritan
Posts: 30
Joined: Sat Feb 13, 2010 4:29 am

Re: Timetrex get punches from Access Control memmory limit

Post by dritan »

Ok, I run setup again and this time it opens the interface!
Thanks for your help!

Dritan
Locked