Project

General

Profile

Bug #1247

ESnet SNMP MA does not scale

Added by Maxim Grigoriev over 8 years ago. Updated over 7 years ago.

Status:
Assigned
Priority:
High
Assignee:
Category:
Production infrastructure
Start date:
04/25/2011
Due date:
% Done:

0%

Estimated time:
Duration:

Description

The SNMAP MA is a single bottleneck for the data retrieval.
Here is my testing results:
(parallel requests, succeeded, attempts)
----------------------------------------
10, 595, 602
20, 583, 602
30, 572, 602
40, 571, 602
50, 587, 602
60, 589, 602
70, 589, 602
80, 589, 602
90, 588, 602
100, 585, 602
110, 582, 602
120, 574, 602
130, 557, 602
140, 526, 602
150, 490, 602
160, 443, 602
170, 254, 602
180, 239, 602
190, 251, 602
200, 252, 602
-------------
Tthe 100 requests is our bottom line and 1000 requests looks
like an unrealistic target right then lets try to have have 100% success with at least 200 requests.

History

#1 Updated by Maxim Grigoriev over 8 years ago

More tests, with increasing time period from 1 day up - single request a time, 100 repetitions:
Notes: Attempted number should be 1 less than shown and Time is the total sum in seconds,
one can get an average by Time/Attempted.
----------------------------------------------------
"Days", "Passed", "Attempted, Time"
1, 101, 102, 117
2, 101, 102, 246
3, 101, 102, 357
4, 101, 102, 457
5, 101, 102, 548
6, 101, 102, 641
7, 101, 102, 724
8, 101, 102, 822
9, 101, 102, 894
10, 101, 102, 988
11, 101, 102, 1074
12, 101, 102, 1157
13, 101, 102, 1246
14, 101, 102, 1309
15, 101, 102, 1379
16, 101, 102, 1449
17, 101, 102, 1542
18, 101, 102, 1613
19, 101, 102, 1698
20, 101, 102, 1784
21, 101, 102, 1855
22, 101, 102, 1944
23, 99, 102, 1986
24, 101, 102, 2106
25, 101, 102, 2182
26, 101, 102, 2345
27, 101, 102, 2446
28, 101, 102, 2487
29, 101, 102, 2622
30, 101, 102, 2581
31, 101, 102, 2657
32, 101, 102, 2544
33, 101, 102, 2644
34, 101, 102, 2739
35, 101, 102, 2773
36, 101, 102, 2833
37, 101, 102, 2932
38, 101, 102, 3014
39, 101, 102, 3085
40, 101, 102, 3174
41, 101, 102, 3250
42, 101, 102, 3347
43, 101, 102, 3419
44, 101, 102, 3505
45, 101, 102, 3591
46, 101, 102, 3684
47, 101, 102, 3777
48, 101, 102, 3863
49, 101, 102, 3960
50, 101, 102, 4019
51, 101, 102, 4137
52, 101, 102, 4222
53, 101, 102, 4451
54, 101, 102, 4711
55, 101, 102, 4618
56, 101, 102, 4629
57, 101, 102, 4661
58, 101, 102, 4761
59, 101, 102, 4837
60, 84, 102, 4088
61, 101, 102, 5009
62, 101, 102, 5105
63, 101, 102, 5188
64, 101, 102, 5260
65, 101, 102, 5362
66, 101, 102, 5450
67, 101, 102, 5544
68, 101, 102, 5634
69, 99, 102, 5627
70, 101, 102, 5946
71, 101, 102, 6146
72, 101, 102, 6184
73, 101, 102, 6124
--------------------
Notice "Passed" numbers less than 101 - that means requests failed those times. Our goal is to eliminate those failures completely for the single request mode.

#2 Updated by Maxim Grigoriev over 8 years ago

We are targeting 100 requests per second for normal "use" created by E-Center and less than 10 seconds response for any request ( up to the year of data). For site centric view it may require to handle 150-200 requests per second because that view based on showing the worst utilization ( bottleneck ) of the "out" interface from all available tracerouts to and from some DOE lab ( 2x10(hops)x10(labs) = 200 ).

#3 Updated by Maxim Grigoriev about 8 years ago

  • Target version changed from v1.0: production to v1.1 (Sept 2011 project review)

#4 Updated by Maxim Grigoriev over 7 years ago

  • Target version changed from v1.1 (Sept 2011 project review) to Year 3 milestone - July-Sep 2012


Also available in: Atom PDF