Sprint on project RakeSearch from 07/26/2018 22:00 (UTC) to 07/29/2018 21:59 (UTC)

Message boards : Number crunching : Sprint on project RakeSearch from 07/26/2018 22:00 (UTC) to 07/29/2018 21:59 (UTC)
Message board moderation

To post messages, you must log in.

Previous · 1 · 2

AuthorMessage
xii5ku

Send message
Joined: 18 Apr 18
Posts: 8
Credit: 14,660,530
RAC: 0
Message 519 - Posted: 29 Jul 2018, 23:15:45 UTC
Last modified: 29 Jul 2018, 23:46:03 UTC

The final results are up at http://formula-boinc.org/sprint.py?sprint=12&year=2018.

Edit: I missed another stats update at the Formula BOINC site when I fetched the data for the following numbers and graphs. For instance, Crunching@EVGA scored a little bit higher than Czech National Team, contrary to the order in my graphs.

All teams together received 62,504,618 credits (+29,819,660 during the last 24 hours).

League 1: 33,369,253 credits total (+15,204,653 in the last 24 hours)
League 2: 28,090,630 credits total (+14,033,559 in the last 24 hours)
League 3: 1,044,735 credits total (+581,448 in the last 24 hours)

Top 30:


Top 30, zoomed in into < 2.5 M points:


League 1 teams: continuous lines
League 2 teams: dotted lines
League 3 teams: dashed lines
ID: 519 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
hoarfrost
Volunteer moderator
Project administrator
Project developer
Project tester
Volunteer developer
Volunteer tester
Project scientist
Help desk expert

Send message
Joined: 11 Aug 17
Posts: 645
Credit: 22,430,895
RAC: 12,763
Message 520 - Posted: 30 Jul 2018, 3:45:49 UTC - in response to Message 517.  
Last modified: 30 Jul 2018, 3:59:45 UTC

We parallelized the validator into 8 instances and restarted the server. It went well.

Problem on your server is probably disk IO and not number of validators.
We can discuss this via email.

yoyo_rkn thank you for information about settings!
In our case increase of parallelism of validator change the situation dramatically due to more complete utilization of "I/O system" and shifting the balance of request to validators side. Of course, site performance has fallen down, tasks reporting and requesting - become slowly. But by 22:00 UTC most of tasks were processed. Natalia - is a great BOINC-master!

xii5ku, thank you for very nice report!
ID: 520 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
VietOZ

Send message
Joined: 26 Mar 18
Posts: 8
Credit: 247,395,533
RAC: 0
Message 530 - Posted: 30 Jul 2018, 16:02:04 UTC - in response to Message 486.  

Hope, that top lines of Best 10 days by Granted Credit will be overwritten!


Wish granted! :)

Congrats to all the teams !! With all the extreme heat around the world and we still manage to broke some records. Hell of a job.
ID: 530 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Millenium

Send message
Joined: 27 Jun 18
Posts: 47
Credit: 9,875,775
RAC: 0
Message 531 - Posted: 30 Jul 2018, 18:12:29 UTC

Oh this explain why the website was so slow in the past days, the server was very very busy! Great job everyone for doing so much work!
ID: 531 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
hoarfrost
Volunteer moderator
Project administrator
Project developer
Project tester
Volunteer developer
Volunteer tester
Project scientist
Help desk expert

Send message
Joined: 11 Aug 17
Posts: 645
Credit: 22,430,895
RAC: 12,763
Message 537 - Posted: 4 Aug 2018, 17:13:56 UTC

Hello folks!

We published a race report and made medals (in addition to badges) for prize-winners. Also we work under small technical review about server works, tasks etc. of this competition. Hope, that this report will cheer you up and will be a memorable event for winners!

xii5ku, can we use your diagrams for information about our projects and distributed computing?

Thank you folks!
ID: 537 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
hoarfrost
Volunteer moderator
Project administrator
Project developer
Project tester
Volunteer developer
Volunteer tester
Project scientist
Help desk expert

Send message
Joined: 11 Aug 17
Posts: 645
Credit: 22,430,895
RAC: 12,763
Message 550 - Posted: 12 Aug 2018, 8:47:14 UTC

Dear folks!

We would like to express our acknowledgment for your participation in Formula BOINC Sprint, and we present a "technical review" of this competition that was held from 26 to 29 of July.

Firstly, some numbers:
During the sprint 1 043 154 valid results were received - that is equivalent by work to 521 577 completed workunits;
Average results receiving rate during the sprint: 14 488 results/hour;
Usual results receiving rate: 2 085 results/hour;
Factor of difference: 14 488 / 2 085 = 6.95;
Saved 6.95 * 3 = 20.85 days. About 3 weeks!

Secondly, we show two diagrams:

1. Results receiving rate before the sprint, during the sprint and after the sprint. Each column on the diagram shows the number of results received during the corresponding hour. We see large spikes immediately after the race start and on the start of the last day of competition.


2. The second diagram looks similar by form with the first one but has another and interesting sense - CPU time spent on computation of results received in the corresponding hour! If we express this time in hours (like on the diagram) and divide this value on 1 hour (because each column corresponds to 1 hour) then we get an average number of cores that processed the results received in this hour! And we see spike values about 27 000 and 28 000 and many values near or higher than 10 000!


But with other numbers situation stays significantly more pecular! Diagrams show that the number of results received in the second half of the sprint is more than in the first half. Not dramatically, but more. If wee look at the daily credit on BOINC Stats we see:

2018-07-25 - 5 944 127
2018-07-26 - 10 473 971
2018-07-27 - 16 968 606
2018-07-28 - 17 614 109
2018-07-29 - 43 448 490
2018-07-30 - 14 594 874

55% of all "sprint credit" where granted on the last day, 29th of July. And really, most of them where granted in the last 12 hours of the sprint - 44% of all sprint credit!
What happened and why?

Each workunit has an attribute to store the next check of its state. In the first two days of the sprint, when the results were received, for many workunits this attribute was not set to appropriate value and remained set to a deadline time. But for many workunits the situation was absolutely normal. After a signal from kashi we found these workunits and set this attribute to a needed value. This was about 168 000 workunits (about 1/3 of total workunits of the sprint!) Also, crunchers continued sending results from their computers to the project server as usual, and performance of existing two validators was exhausted. In 12 hours until the end of the sprint, the necessity of increasing the number of validators was clear and we made a restart of project server with eight validators. Results vaidation rate increased 4 times, and in the last 12 hours 44% of total workunits and results of the entire sprint were processed with an average rate of 18.35 times higher than usual and 2.5 times higher than average in the sprint!

And we come to question of "project killing". When project runs with eight validators, many pages of the project website opened too slowly, but when we look at this situation from the project server, the dramaticity downs to zero. Yes, project server processed about 8 workunits (and 16 results) per second. Yes, message boards opened with timeouts or lags like tens of seconds. But the server load by CPU was about only one core, I/O load - significantly higher, but the lag of command execution was zero and server work was under full control. At any time we could simply change settings and make a relatively fast restart of the project server (about 1-2 minutes). Moreover, some further optimizations of the project server processes, database and data archivation can significantly speedup its work. (Some of them are already implemented).

And it would be very interesting to check it out!
ID: 550 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
mmonnin

Send message
Joined: 8 Sep 17
Posts: 22
Credit: 19,171,868
RAC: 12,035
Message 554 - Posted: 13 Aug 2018, 9:54:03 UTC

Were these when tasks became validated or either of the two tasks per unit? The long tail would make me assume it was when tasks were validated and thus push a higher completion count towards the 2nd half of the event.
ID: 554 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
hoarfrost
Volunteer moderator
Project administrator
Project developer
Project tester
Volunteer developer
Volunteer tester
Project scientist
Help desk expert

Send message
Joined: 11 Aug 17
Posts: 645
Credit: 22,430,895
RAC: 12,763
Message 555 - Posted: 13 Aug 2018, 20:21:44 UTC - in response to Message 554.  
Last modified: 13 Aug 2018, 20:25:53 UTC

Were these when tasks became validated or either of the two tasks per unit? The long tail would make me assume it was when tasks were validated and thus push a higher completion count towards the 2nd half of the event.

On charts above, results broken by received time. If "first result" were received July 27th and "second" - July 28th each of them will fall on its day. But results must be validated! For example if "second" result does not completed and "first", consequently - can not be verified yet, both of results will not be counted on this chart. But when we made this charts (~ 3 days after sprint) almost all workunits - get a quorum and it's results were validated.
ID: 555 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
xii5ku

Send message
Joined: 18 Apr 18
Posts: 8
Credit: 14,660,530
RAC: 0
Message 592 - Posted: 28 Sep 2018, 7:26:06 UTC - in response to Message 537.  

hoarfrost wrote:
xii5ku, can we use your diagrams for information about our projects and distributed computing?

Sorry for responding late. Yes, you may use these as you like.

Thanks once more for paying a lot of attention to the server during the sprint, and thanks for your own interesting reports.
ID: 592 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
hoarfrost
Volunteer moderator
Project administrator
Project developer
Project tester
Volunteer developer
Volunteer tester
Project scientist
Help desk expert

Send message
Joined: 11 Aug 17
Posts: 645
Credit: 22,430,895
RAC: 12,763
Message 593 - Posted: 29 Sep 2018, 7:40:13 UTC - in response to Message 592.  

hoarfrost wrote:
xii5ku, can we use your diagrams for information about our projects and distributed computing?

Sorry for responding late. Yes, you may use these as you like.

Thank you! They can be useful for us!
ID: 593 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Previous · 1 · 2

Message boards : Number crunching : Sprint on project RakeSearch from 07/26/2018 22:00 (UTC) to 07/29/2018 21:59 (UTC)

©2024 The searchers team, Karelian Research Center of the Russian Academy of Sciences