Tasks getting to 100% and then keep going from 0%

Message boards : Number crunching : Tasks getting to 100% and then keep going from 0%
Message board moderation

To post messages, you must log in.

AuthorMessage
Profile PDW

Send message
Joined: 8 Sep 17
Posts: 34
Credit: 100,058,938
RAC: 8
Message 7 - Posted: 9 Sep 2017, 9:09:07 UTC

Hi,

I have just noticed that some tasks get to 100% completed and then keep going again from 0% They have been running for quite a while now. Should this be happening ?



Have completed 3 other tasks in about 13 - 14 hours.
ID: 7 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Natalia
Volunteer moderator
Project administrator
Project developer
Project tester
Volunteer developer
Volunteer tester
Project scientist
Help desk expert

Send message
Joined: 11 Aug 17
Posts: 103
Credit: 1,973,929
RAC: 15
Message 9 - Posted: 9 Sep 2017, 15:52:53 UTC - in response to Message 7.  

Dear PDW, could you post the contents of a checkpoint.txt file from the slot of that task?
ID: 9 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile PDW

Send message
Joined: 8 Sep 17
Posts: 34
Credit: 100,058,938
RAC: 8
Message 15 - Posted: 9 Sep 2017, 19:38:02 UTC - in response to Message 9.  

Hi Natalia,

Several tasks that went past 100% have completed, although they ran for several more hours their final percentage only got up to about 1%.

Still have one task that has now run for 29.7 hours, task name is: R9_000000124

Here is the current contents of the checkpoint.txt file in its slot:

# Move search of pairs OLDS status

# Generation of DLS status

9
{
0 1 2 3 4 5 6 7 8 
4 2 8 7 0 1 3 6 5 
5 3 1 8 7 2 0 4 6 
2 6 0 4 8 7 1 5 3 
8 0 6 5 3 4 7 1 2 
7 4 3 2 5 6 8 0 1 
6 7 4 1 2 8 5 3 0 
3 5 7 6 1 0 2 8 4 
1 8 5 0 6 3 4 2 7 
}
56

1 2 
1 3 
1 4 
1 5 
1 6 
1 8 
2 0 
2 1 
2 3 
2 4 
2 5 
2 7 
2 8 
3 0 
3 1 
3 2 
3 4 
3 6 
3 7 
3 8 
4 0 
4 1 
4 2 
4 3 
4 5 
4 6 
4 7 
4 8 
5 0 
5 1 
5 2 
5 4 
5 6 
5 7 
5 8 
6 0 
6 1 
6 3 
6 4 
6 5 
6 7 
6 8 
7 0 
7 2 
7 3 
7 4 
7 5 
7 6 
7 8 
8 1 
8 2 
8 3 
8 4 
8 5 
8 6 
8 7 

1 2 -1
8 7 55

0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 

0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 

0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 

0 1 1 1 1 1 1 1 1 
1 1 0 0 0 1 1 1 1 
1 1 1 1 1 1 0 1 1 
1 0 0 1 1 0 1 1 1 
1 0 1 1 1 1 1 1 1 
1 1 1 0 1 1 1 0 1 
1 1 0 0 1 0 1 1 0 
1 0 1 0 1 0 1 1 1 
1 1 1 0 1 1 1 1 1 

1 0 1 1 1 1 1 1 1 
1 1 1 0 1 0 1 1 1 
1 1 0 1 1 1 1 1 1 
1 1 1 1 0 1 0 1 1 
1 1 1 0 1 1 1 0 1 
1 1 1 1 0 1 1 1 0 
1 1 1 0 1 1 1 1 1 
1 1 1 1 0 1 1 1 1 
0 1 1 1 1 1 1 1 1 

1 1 0 1 1 1 1 1 1 
1 0 1 1 1 1 1 1 1 
0 1 1 1 0 0 1 1 1 
0 1 1 1 1 1 1 1 1 
1 1 1 1 1 1 0 1 0 
1 1 1 0 1 1 1 1 1 
1 1 1 1 0 1 1 1 1 
1 1 1 1 1 1 0 1 1 
1 1 1 1 1 1 1 0 1 

1 1 1 0 1 1 1 1 1 
0 1 0 1 1 1 0 1 1 
0 0 1 1 1 1 1 1 1 
1 1 1 1 1 1 1 0 0 
1 1 1 1 0 1 1 1 1 
0 1 0 1 1 1 1 1 1 
0 1 1 1 1 0 1 0 1 
0 1 1 1 1 1 1 1 1 
1 1 1 1 1 0 1 1 1 

1 1 1 1 0 1 1 1 1 
0 1 1 1 1 1 1 1 1 
1 1 1 1 1 1 1 0 1 
1 1 1 0 1 1 1 1 1 
1 1 1 1 1 0 1 1 1 
1 0 1 1 1 1 1 1 1 
1 1 0 1 1 1 1 1 1 
1 0 1 1 1 1 1 1 0 
1 1 1 1 1 1 0 1 1 

1 1 1 1 1 0 1 1 1 
1 1 0 0 1 1 1 1 0 
0 1 1 1 1 1 1 1 1 
1 1 1 1 0 1 1 0 1 
1 1 0 0 1 1 1 1 1 
1 1 1 1 0 1 1 1 1 
1 1 1 1 1 1 0 1 1 
1 0 1 1 1 1 1 1 1 
1 1 0 1 1 1 1 1 1 

1 1 1 1 1 1 0 1 1 
1 1 1 1 1 1 1 0 1 
1 1 1 0 0 1 1 1 0 
1 0 1 1 1 1 1 1 1 
0 1 0 1 1 1 1 1 1 
1 1 1 1 1 0 1 1 1 
0 1 1 1 1 1 1 1 1 
1 1 1 0 1 1 1 1 1 
1 1 1 1 0 1 1 1 1 

1 1 1 1 1 1 1 0 1 
1 1 0 0 1 1 1 1 1 
1 1 1 1 0 1 1 1 1 
1 1 1 1 1 0 1 1 1 
0 1 1 1 1 1 0 1 1 
0 1 1 1 1 1 1 1 1 
1 0 1 1 1 1 1 1 1 
1 1 0 1 1 1 1 1 1 
1 1 1 1 1 1 1 1 0 

1 1 1 1 1 1 1 1 0 
1 1 0 1 1 1 1 1 1 
1 1 1 0 1 1 1 1 1 
1 1 1 1 0 1 1 1 1 
0 1 1 1 1 1 1 1 1 
1 1 1 1 1 1 0 1 1 
1 1 1 1 1 0 1 1 1 
1 1 1 1 1 1 1 0 1 
1 0 1 1 1 1 1 1 1 


516000000

# Move search component status

0 0 0
0 516000000


Thanks.
ID: 15 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile [B@P] Daniel

Send message
Joined: 8 Sep 17
Posts: 99
Credit: 402,603,726
RAC: 0
Message 21 - Posted: 10 Sep 2017, 10:22:54 UTC
Last modified: 10 Sep 2017, 10:24:13 UTC

This display issue happened to me too, and it is very misleading. If you cannot estimate when task would end, better approach would be to display 99.xxx% and increase fraction part only.
ID: 21 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
hoarfrost
Volunteer moderator
Project administrator
Project developer
Project tester
Volunteer developer
Volunteer tester
Project scientist
Help desk expert

Send message
Joined: 11 Aug 17
Posts: 645
Credit: 22,428,188
RAC: 12,795
Message 23 - Posted: 11 Sep 2017, 7:44:18 UTC - in response to Message 15.  

Still have one task that has now run for 29.7 hours, task name is: R9_000000124

Here is the current contents of the checkpoint.txt file in its slot:
...

PWD, thank you for report. Did the calculations succeed?
if I understand correctly, this task is very large. Similar tasks are very rare.
ID: 23 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile PDW

Send message
Joined: 8 Sep 17
Posts: 34
Credit: 100,058,938
RAC: 8
Message 24 - Posted: 11 Sep 2017, 8:26:41 UTC - in response to Message 23.  

Still have one task that has now run for 29.7 hours, task name is: R9_000000124

Here is the current contents of the checkpoint.txt file in its slot:
...

PWD, thank you for report. Did the calculations succeed?
if I understand correctly, this task is very large. Similar tasks are very rare.

Hi,

Yes, all the tasks that went from 100% to 0% completed successfully :-)
Only 2 tasks have failed, they were 32 bit tasks and I did not have the 32 libraries installed, I do now !
ID: 24 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile [B@P] Daniel

Send message
Joined: 8 Sep 17
Posts: 99
Credit: 402,603,726
RAC: 0
Message 28 - Posted: 12 Sep 2017, 9:46:49 UTC

I also had few very long tasks - the longest one needed 2 days 2.5 hours to complete. Two of them are validated successfully, 3rd one waits for validation.

http://rake.boincfast.ru/rakesearch/result.php?resultid=26211
http://rake.boincfast.ru/rakesearch/result.php?resultid=26180
http://rake.boincfast.ru/rakesearch/result.php?resultid=26160
ID: 28 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
hoarfrost
Volunteer moderator
Project administrator
Project developer
Project tester
Volunteer developer
Volunteer tester
Project scientist
Help desk expert

Send message
Joined: 11 Aug 17
Posts: 645
Credit: 22,428,188
RAC: 12,795
Message 31 - Posted: 12 Sep 2017, 12:36:20 UTC - in response to Message 28.  

I also had few very long tasks - the longest one needed 2 days 2.5 hours to complete. Two of them are validated successfully, 3rd one waits for validation.

http://rake.boincfast.ru/rakesearch/result.php?resultid=26211
http://rake.boincfast.ru/rakesearch/result.php?resultid=26180
http://rake.boincfast.ru/rakesearch/result.php?resultid=26160


Daniel, yes it is. These tasks are about 2 - 2.5 times larger than "usual task".

Thank you!
Happy crunching!
ID: 31 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Supracore

Send message
Joined: 29 Oct 17
Posts: 2
Credit: 523,379
RAC: 1,425
Message 141 - Posted: 1 Nov 2017, 16:33:29 UTC

Some tasks stay at 100% and never ends until hours. Is that normal?
ID: 141 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
hoarfrost
Volunteer moderator
Project administrator
Project developer
Project tester
Volunteer developer
Volunteer tester
Project scientist
Help desk expert

Send message
Joined: 11 Aug 17
Posts: 645
Credit: 22,428,188
RAC: 12,795
Message 143 - Posted: 1 Nov 2017, 20:41:54 UTC - in response to Message 141.  
Last modified: 1 Nov 2017, 20:45:38 UTC

Some tasks stay at 100% and never ends until hours. Is that normal?

Supracore, hello!

Some tasks can be a very large - approximately 2 times large than usual task. We can not precise predict a task size and in this case progress can reach 100% within a few hours before actual finish of computation.

Than you for participation in our project!
ID: 143 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Supracore

Send message
Joined: 29 Oct 17
Posts: 2
Credit: 523,379
RAC: 1,425
Message 144 - Posted: 1 Nov 2017, 20:57:20 UTC

Thx for the info. :-)
ID: 144 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
blitzzkreeg

Send message
Joined: 10 Oct 19
Posts: 2
Credit: 4,160,220
RAC: 1,028
Message 1209 - Posted: 9 Nov 2019, 6:18:55 UTC

I've had a similar issue, in this case however tasks reach and remain at 100% but keep going while never displaying "Ready to Report".
Not sure if such tasks break and require an abort or if I should let them run and somehow they will complete.
Here's a sample task still going for over 14hrs and counting...

Application
RakeSearch for rank 10 3.01
Name
rake_r10_0000000000205778
State
Running
Received
11/8/2019 10:55:01
Report deadline
11/15/2019 11:01:17
Estimated computation size
100,000 GFLOPs
CPU time
13:55:15
CPU time since checkpoint
00:00:07
Elapsed time
14:00:37
Estimated time remaining
---
Fraction done
100.000%
Virtual memory size
3.16 MB
Working set size
5.51 MB
Directory
slots/11
Process ID
33276
Executable
rakesearch10_3.1_windows_x86_64.exe
ID: 1209 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
hoarfrost
Volunteer moderator
Project administrator
Project developer
Project tester
Volunteer developer
Volunteer tester
Project scientist
Help desk expert

Send message
Joined: 11 Aug 17
Posts: 645
Credit: 22,428,188
RAC: 12,795
Message 1210 - Posted: 9 Nov 2019, 6:39:38 UTC - in response to Message 1209.  

I've had a similar issue, in this case however tasks reach and remain at 100% but keep going while never displaying "Ready to Report".
...
Directory slots/11
...

Could you check a file slots/11/checkpoint.txt, if the task still in progress? If it changes every 1 .. 5 minutes and number of processed squares int last lines of the file increased - it is a simply large task.
ID: 1210 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
blitzzkreeg

Send message
Joined: 10 Oct 19
Posts: 2
Credit: 4,160,220
RAC: 1,028
Message 1211 - Posted: 9 Nov 2019, 8:09:56 UTC - in response to Message 1210.  

Hi hoarfrost,

It appears indeed that the process is ongoing and value increasing every min after checking slot/11 .txt file, your feedback is well appreciated and reassuring! :)

1 1 1 1 1 1 1 1 1 0
1 1 1 0 1 1 1 1 1 1
0 1 1 1 1 1 1 1 1 1
1 1 1 1 0 1 1 1 1 1
1 1 1 1 1 1 0 1 1 1
1 1 0 1 1 1 1 1 1 1
1 1 1 1 1 1 1 0 1 1
1 0 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 0 1
1 1 1 1 1 0 1 1 1 1


1358000000

0 114 114


1 1 1 1 1 1 1 1 1 0
1 1 1 0 1 1 1 1 1 1
0 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 0 1 1
1 0 1 1 1 1 1 1 1 1
1 1 0 1 1 1 1 1 1 1
1 1 1 1 0 1 1 1 1 1
1 1 1 1 1 1 0 1 1 1
1 1 1 1 1 1 1 1 0 1
1 1 1 1 1 0 1 1 1 1


1360000000

0 114 114

1 1 1 1 1 1 1 1 1 0
1 1 1 0 1 1 1 1 1 1
0 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 0 1 1
1 1 1 1 1 1 0 1 1 1
1 0 1 1 1 1 1 1 1 1
1 1 0 1 1 1 1 1 1 1
1 1 1 1 0 1 1 1 1 1
1 1 1 1 1 1 1 1 0 1
1 1 1 1 1 0 1 1 1 1


1363571201

1 115 115
ID: 1211 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote

Message boards : Number crunching : Tasks getting to 100% and then keep going from 0%

©2024 The searchers team, Karelian Research Center of the Russian Academy of Sciences