Useful things for continuing the report

todo:

finish document under:
/afs/cern.ch/user/d/dfrodrig/workspace/docs/reports

Add more results to *.sxc :
/afs/cern.ch/user/d/dfrodrig/workspace/docs/testResults
Go get them at:
/afs/cern.ch/user/d/dfrodrig/workspace/GMAN/tests/results

lxb6117, lxb6118 configurations also under ~/workspace/docs/reports/tmp!

Command for running consumer: tail -n 100 -f cons | nc lxb6118 6163 > /tmp/tmp

and not referring to the report:
:: http://lxb6118:8161/admin/topics.jsp

Getting this:

Exception in thread "ActiveMQ Transport Server: ssl://localhost:61617" java.lang.OutOfMemoryError: GC overhead limit exceeded
at java.io.ByteArrayOutputStream.(ByteArrayOutputStream.java:60)
at com.sun.net.ssl.internal.ssl.OutputRecord.(OutputRecord.java:56)
at com.sun.net.ssl.internal.ssl.OutputRecord.(OutputRecord.java:66)
at com.sun.net.ssl.internal.ssl.AppOutputStream.(AppOutputStream.java:33)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.init(SSLSocketImpl.java:519)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.(SSLSocketImpl.java:433)
at com.sun.net.ssl.internal.ssl.SSLServerSocketImpl.accept(SSLServerSocketImpl.java:255)
at org.apache.activemq.transport.tcp.TcpTransportServer.run(TcpTransportServer.java:153)
at java.lang.Thread.run(Thread.java:619)

From time to time... I'll have to carefully analyse.


right now, I am running the tests veryfing saturation and before saturation. Up to now:

	MessageSize	Nmessages	Speed	Saturation	SpeedAfterSaturation
1000B			16384	8822.15 [~1.85s]	15 Messages	334M/s[~49s]
	8192	9102 m/s[~0.9s]	20 Messages	334M/s [~24.5s]
	32768	8936 m/s[~3.6s]	6 Messages	334M/s [~98s]

Configuration at lxb6118

James installed Activemq 5.0 (latest release )

configuration files are under

/etc/activemq

activemq-wrapper.conf (for configuring the activemq wrapping)
activemq.xml (for configuring the activemq broker)
log4j.properties

logs/journal are under /var/lib/activemq

and stopping starting has a dedicated start script :

service activemq start|stop

lxb6117 will keep activemq4.1.1

Brief notes on remake of the tests

Additional variables to take into account:

Maybe it' snot the same in 5.0

Be careful (for the future...

> DO NOT USE UNDERSCORES IN YOUR BROKERNAMES.
>

http://mail-archives.apache.org/mod_mbox/activemq-users/200707.mbox/%3C84CFFBCA-88FA-4366-A38D-11720C28010B@gmail.com%3E

meeting with James and Max

Just to have a milestone and work on the coming tasks.

- I presented the work done up to know, briefly explaining what has been done in terms of the scripts using scripting ( based on netcat and wassh.)

- on doubts expressed by me and Max regarding the context of the messaging, James clarified the following points:
a) OGF hasn't still endorsed any messaging structure within the grid: all grid is built on point to point;
b) The messaging system is supposed to be a thin layer. Main idea is to decouple the services from knowing what he has to contact;
c) Some use cases for the moment are:
i) SAM publication: workerNodes/Clients are located anywhere. SAM has a central service, getting all the reports from these test Jobs, but some of them are deployed locally. The idea is that the test results would be sent to the closest broker and this would send the message to the proper Central SAM service.

Summing up today

I already was able to produce some nice graphs, but still depending a lot on manual editing, instead of auto-generation. Problem is, I first need to output to CVS, and take care of all the strange things in the spreadsheet.

I was looking at the possibility of using a graphs tool, and I found it, but it's not immediate to make the transformation, and particularly make some transformations all around. maybe Excel 2003 has a better support? The aim of the project is *not* transforming.

Anyway, today, produced:
under /GMAN/tests/shell:
> gmanNcTestRun.sh #creates batch messages and sends them. configurable inside (report to previous posts)
> runSomeTests. good ones were : timedResults{21,23,25}.log[.xml] this were already outputed to csvTestResults.csv, and edited in csvTestResults.sxc I might delete all the rest? :S
> test.xslt. Transformation of the current output to a different one (according to the software downloaded: http://www.maani.us/xml_charts/index.php?menu=Tutorial&submenu=Chart_Data ). charts.swf seems a proper soft for producing charts, but have to work out the transformations :(

under project SmallUtils:
A simple SaxParser that converts the log files produced to csv, and appends all together in csvTestResults.csv.



Some ideas for next tests:
1. Measure broker disk consumption, and try to push it to 100% (maybe increase message payload size? )
2. test with many producers;
3. test with consumers.

for automation, we'll need to have scripts allowing to subscribe and unscribe (use wassh?)

and that's all folks!