Tag: Programming
WATCH – Run periodically a program
You have enough of typing every time ls -lah to see the state of copying files? Try watch ls -lah. Watch runs the given command in a configurable interval and displays the result. It can also highlight the diff between two outputs.
Debian Bug Leaves Private SSL/SSH Keys Guessable
Ganglia
I had to install Ganglia on ten boxes. I downloaded the source package and followed the readme file. It helped me very much until the point I got on all ten boxes gmond to run and on one the web server. But what now? The web server hadn’t displayed any information. So I came to the point I need gmetad to collect all the informations. I putted it on the web server box to the gmond start script and added the configuration file. First you have to change the gmetad port because the default installation has the same as gmond configured. Then you have to set all boxes in the field data_source like
data_source "cluster name" box1 box2 box3 box4 box5
then set the grid_name like
gridname "grid name"
The last step is to set the grid name also in the gmond configuration like
cluster {
name = "cluster name"
...
}
After these last steps Ganglia and the web interface are running well.
Summary for a working Ganglia with GUI:
- run gmond on every server
- set the grid name in gmond.conf
- run gmetad on the same server as the web interface
- set all boxes that run gmond in gmetad.conf
- set the cluster name in gmetad.conf
The tool Cssh helped me very much on doing the installation on all ten boxes in parallel.
Python Is Not Java dirtSimple.org
Eclipse Test & Performance Tools Platform Project troubles
After using TPTP for about a half year for my bachelor I am at a point where I would remove it. But I don’t have to decide it. I reported so far 6 bugs. Everyone of them detains me on profiling. I can create single traces but if I repeat tracing I am in trouble. JVM crashes, 100% CPU usages and so on. I have now written my own “profiler” two classes that give me the covered lines back of a profiled program. It uses the Java debugging interface (JDI).
The first class is called Agent that starts a event catcher thread (the debugging agent) if it is called without any arguments. This debugging agent waits for a connecting JVM. With a argument it calls a sample method to test profiling. You have to supply the debugging agent port and host like -agentlib:jdwp=transport=dt_socket,suspend=y,address=localhost:55555. The other class is EventCatcherThread which enables the single step event and waits for a connecting JVM to profile a program. Only 278 lines of java code are necessary.
Agent.java
import java.util.ArrayList;
import java.util.Iterator;
import java.util.List;
import com.sun.jdi.connect.IllegalConnectorArgumentsException;
import com.sun.jdi.connect.VMStartException;
public class Agent {
private ArrayList<String> _linesPassed = new ArrayList<String>();
public void addSourceCodeLine() {
_linesPassed.add("bla");
}
public List<String> getLinesPassed() {
return _linesPassed;
}
public static void debug() throws Exception {
List<String> passedLine = new ArrayList<String>();
EventCatcherThread eventThread = EventCatcherThread.connectToVm(55555);
eventThread.setPassedCodeLines(passedLine);
eventThread.start();
eventThread.resumeVM();
eventThread.join();
for (Iterator<String> iterator = passedLine.iterator(); iterator
.hasNext();) {
String lineKey = (String) iterator.next();
System.out.println(lineKey);
}
}
public static void main(String[] args) throws Exception,
IllegalConnectorArgumentsException, VMStartException {
if (args.length != 0) {
debug();
} else {
Agent agent = new Agent();
agent.traceTest();
}
}
public void traceTest() {
String string = new String();
string = string + " fdgvdf ";
string = string.concat("bla");
System.out.println(string);
}
}
EventCatcherThread.java
package org.projectory.ezunit;
import java.io.IOException;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import org.eclipse.jdi.Bootstrap;
import com.sun.jdi.AbsentInformationException;
import com.sun.jdi.ThreadReference;
import com.sun.jdi.VMDisconnectedException;
import com.sun.jdi.VirtualMachine;
import com.sun.jdi.connect.AttachingConnector;
import com.sun.jdi.connect.IllegalConnectorArgumentsException;
import com.sun.jdi.connect.Connector.Argument;
import com.sun.jdi.event.Event;
import com.sun.jdi.event.EventIterator;
import com.sun.jdi.event.EventQueue;
import com.sun.jdi.event.EventSet;
import com.sun.jdi.event.StepEvent;
import com.sun.jdi.event.VMDeathEvent;
import com.sun.jdi.event.VMDisconnectEvent;
import com.sun.jdi.event.VMStartEvent;
import com.sun.jdi.request.EventRequest;
import com.sun.jdi.request.EventRequestManager;
import com.sun.jdi.request.StepRequest;
public class EventCatcherThread extends Thread {
private final VirtualMachine _vm;
private String[] _excludes;
private boolean _connected = true;
private String[] _includes;
private Map<String, Location> _lineStat = new HashMap<String, Location>();
private List<String> _passedCodeLines;
EventCatcherThread(VirtualMachine vm, String[] excludes, String[] includes) {
super("event-handler");
_vm = vm;
_excludes = excludes;
_includes = includes;
}
public void run() {
EventQueue queue = _vm.eventQueue();
List<ThreadReference> allThreads = _vm.allThreads();
for (ThreadReference thread : allThreads) {
if (thread.uniqueID() == 1) {
enableSingleStepEvent(thread);
}
}
while (_connected) {
try {
EventSet eventSet = queue.remove();
EventIterator it = eventSet.eventIterator();
while (it.hasNext()) {
Event nextEvent = it.nextEvent();
handleEvent(nextEvent);
}
eventSet.resume();
} catch (InterruptedException exc) {
// Ignore
} catch (VMDisconnectedException discExc) {
handleDisconnectedException();
break;
}
}
}
private void enableSingleStepEvent(ThreadReference thread) {
EventRequestManager mgr = _vm.eventRequestManager();
StepRequest req = mgr.createStepRequest(thread, StepRequest.STEP_LINE,
StepRequest.STEP_INTO);
req.setSuspendPolicy(EventRequest.SUSPEND_EVENT_THREAD);
for (int i = 0; i < _excludes.length; ++i) {
req.addClassExclusionFilter(_excludes[i]);
}
for (int i = 0; i < _includes.length; ++i) {
req.addClassFilter(_includes[i]);
}
req.enable();
}
private void singleStepEvent(StepEvent event) {
if (_passedCodeLines != null) {
try {
final String sourcePath = event.location().sourcePath();
final int lineNumber = event.location().lineNumber();
final String method = event.location().method().toString();
final String lineKey = sourcePath + ":" + method + ":" + lineNumber;
if (_lineStat.containsKey(lineKey)) {
Location location = _lineStat.get(lineKey);
location.countCall();
} else {
Location location = new Location(sourcePath, method, lineNumber);
location.countCall();
_lineStat.put(lineKey, location);
}
if (!_passedCodeLines.contains(lineKey)) {
_passedCodeLines.add(lineKey);
}
} catch (AbsentInformationException e) {
e.printStackTrace();
}
} else {
System.err
.println("Cannot set line numbers, because the container is null!");
}
}
/**
* Dispatch incoming events
*/
private void handleEvent(Event event) {
if (event instanceof StepEvent) {
singleStepEvent((StepEvent) event);
} else if (event instanceof VMStartEvent) {
vmStartEvent((VMStartEvent) event);
} else if (event instanceof VMDisconnectEvent) {
vmDisconnectEvent((VMDisconnectEvent) event);
} else if (event instanceof VMDeathEvent) {
// application exit
} else {
System.out.println("Unhandled event type: " + event.getClass());
}
}
private void handleDisconnectedException() {
EventQueue queue = _vm.eventQueue();
while (_connected) {
try {
EventSet eventSet = queue.remove();
EventIterator iter = eventSet.eventIterator();
while (iter.hasNext()) {
Event event = iter.nextEvent();
if (event instanceof VMDisconnectEvent) {
vmDisconnectEvent((VMDisconnectEvent) event);
}
}
eventSet.resume();
} catch (InterruptedException exc) {
// ignore
}
}
}
/**
* Enable the SingleStepEvent if the VM was started in suspend mode.
* @param event
*/
private void vmStartEvent(VMStartEvent event) {
EventRequestManager mgr = _vm.eventRequestManager();
StepRequest req = mgr.createStepRequest(event.thread(),
StepRequest.STEP_LINE, StepRequest.STEP_INTO);
req.setSuspendPolicy(EventRequest.SUSPEND_EVENT_THREAD);
for (int i = 0; i < _excludes.length; ++i) {
req.addClassExclusionFilter(_excludes[i]);
}
for (int i = 0; i < _includes.length; ++i) {
req.addClassFilter(_includes[i]);
}
req.enable();
}
private void vmDisconnectEvent(VMDisconnectEvent event) {
_connected = false;
}
@SuppressWarnings("unchecked")
public static EventCatcherThread connectToVm(int port) throws Error,
IOException, InterruptedException,
IllegalConnectorArgumentsException {
AttachingConnector c = getConnector();
Map<String, Argument> arguments = c.defaultArguments();
Argument portArg = arguments.get("port");
portArg.setValue("" + port);
// FIXME: localhost doesn't work
Argument hostArg = arguments.get("hostname");
hostArg.setValue("elaste");
VirtualMachine myVM = c.attach(arguments);
myVM.setDebugTraceMode(0);
final String[] excludes = { "org.hibernate.*", "net.sf.cglib.*",
"org.junit.*", "java.*", "javax.*", "sun.*", "com.sun.*" };
final String[] includes = { };
EventCatcherThread eventThread = new EventCatcherThread(myVM, excludes,
includes);
return eventThread;
}
private static AttachingConnector getConnector() {
AttachingConnector result = null;
List<AttachingConnector> allConnectors = Bootstrap
.virtualMachineManager().attachingConnectors();
if (allConnectors.size() > 0) {
result = allConnectors.get(0);
}
return result;
}
public void resumeVM() {
_vm.resume();
}
public List<String> getPassedCodeLines() {
return _passedCodeLines;
}
public void setPassedCodeLines(List<String> codeLines) {
_passedCodeLines = codeLines;
}
public Location getLineStat(String lineKey) {
return _lineStat.get(lineKey);
}
}
ioerror: Chip and Pin attacks
Using the X clipboard from the command line
Mark entity as deleted using Hibernate Discriminator
heise open – IBM-Manager: “OOXML ist eine Sackgasse”
Linux-Magazin Online Conference Streaming
Linux-Magazin Online Conference Streaming Live Stream of FOSDEM ’08 on Februray 23-24.
Ist der BND eine Gruppe von Computerfreaks?
Der BND erklärt auf seinen Seiten Begriffe, die doch eindeutig aus dem Computerbereich stammen, z.B. OSINT oder SIGINT. Also aufpassen wenn man das nächste mal ein Programm per SIGINT beendet. Man könnte mehr bekommen als man will. Ist der Nachichtendienst nur ein Deckmantel für Softwareentwicklung?
Exubero – JUnit Anti-patterns
Otaku, Cedrics weblog: Statistical testing
Namesys things
Namesys vanishes, but Reiser project lives on | Underexposed – CNET News.com
Development issues part 2: Bug tracking [LWN.net]
Time for the annual bugzilla statistics! [LWN.net]
Time for the annual bugzilla statistics! [LWN.net]
Klapper
Amazing bug tracker statistic.
Development issues part 1: Project communication [LWN.net]
Eclipse plugins and the version
Trouble with installed plugins that aren’t update completly if you do not change the version number? But you do not want to set it higher for every test. Then it helps to use the qualifier. Set the version string fo your plugins as follow 1.2.3.qualifier and on deploying disable qualifier replacement with a static string under the options tab. Voila. Eclipse will replace the qualifier string with the current timestamp on every deployment.
“The Law of Software Development and Envelopment at MIT: Every program in d…” – Full Surveillance Mode
oae
22C3: A discussion about modern disk encryption systems
A Collection of JVM Options
A Collection of JVM Options Show also in what version the option exists.
curve to
curve t It is so easy to use cairo.
Hibernate Search
Amazon patent fully revoked: skirmish victory for FFII
Please read the full story here http://press.ffii.org/Press_releases/Amazon_patent_fully_revoked%3A_skirmish_victory_for_FFII
EU tells open source to start paying MS patent tax
Please read the full story here: http://press.ffii.org/Press_releases/EU_tells_open_source_to_start_paying_MS_patent_tax .
Automated evaluation
How to evaluate? First my colleagues should only test it. But my professor thought that this isn’t enough for an expressive and compareable result. So the idea is to set errors and start the evaluation when tests fail. And everything should work automatically. The user start it and after a month or so you have the result without any intervention by the user
Jesting
There exists a tool called Jester that changes the source code of programms and run the tests after every change. If no tests fail it will remember the position and what changed. With this procedure it is possible to find source code that isn’t well tested. But this isn’t exactly what we need. We need failed tests to test how good an evaluator finds the changed method.
EzUnit and Jester
The procedure is as follow. First we get all methods of a projects source folder from the Eclipse (JDT). Then we do one style of change, there are 11 variants of changing the source code, after each other for every method. These variants are taken from Jester. I will explain them later. Then the test suite is started. If compilation is necessary the launch waits for completion and continues thereafter. There are three cases after the test site launch. Junit can give an error back this means something not test specific goes wrong. In the case Junit returns with a failure the evaluation is started because thento reverse there are failed tests. The third case is that nothing failed after the change. Then there is nothing to evaluate like after an error and you know that there is bad tested code in the project. The next step is to take all made changes back. If all methods are changed with all variants of source code changes a csv file is written with all results in the project root folder.
Changing source code
Jester has 11 variant of changing source code. All are implemented by EzUnit. There is an number changer that searches for one digit and add 1 to it. This Jester implementation can lead to compilation errors because it changes an octal number representation from 0x … to 1x… If such a change happens it is ignored. The other methods replace java language constructs to reverse there meaning like “true” to “false”, “if (” to “if (true ||”, “if (” to “if (false &&”, “==” to “!=”, “!=” to “==”, “++” to “−−” and “−−” to “++”. Such changes can lead to endless loops. To avoid an ever blocking test the timeout annotation (@Test(timeout = 60000) for Junit4 should be used. But we saw also endless loops where the cause is unknown for now and where the setting of the timeout are not working. In this cases the launch has an timeout set so that it continues with another change.
The time
First tests show that a evaluation run can take very long dependent on the number of methods and tests a project has and the time for one test run. An evaluation run with commons-codec (191 tests) needs 4 hours on double processor with 1.66GHz and for commons-math (1022 tests) about 4 days.
You must be logged in to post a comment.