AnalysisTools

There are several kinds of analysis tools that operate on Power TAC game logs. One way to do analyses would be to develop a database schema for the data that's contained in a game log, populate the database, and then write queries to extract the data. Nobody has yet worked out such a schema, or if they have they have not contributed it. So here we describe how to extract data from an individual game using a logtool analyzer, and how to iterate a logtool analyzer across a tournament.

Per-game analyzers

The Power TAC simulator produces two log files for each boot session, and two for each sim session.

1. The first is the "trace" log that contains a record of significant activities in the simulator. It's primarily used for debugging, but also sometimes contains information needed for analysis. Since the format of log entries in the trace log is not standardized, data extraction from the trace log requires a certain amount of text processing. There are currently no examples of such analyzers, since it's considered a bug to have important data not appearing in the state log.
2. The second is the "state" log, containing a record of each significant state change in the simulator. These are all object creation and method calls. Not all object creation and method call events are logged, but generally all instances of domain types and messages are logged when they are created, as are method calls that change their state. The format of this log is standardized by the use of Java annotations to trigger the logging behavior. This means that a single codebase (it's in the powertac-aop module inside powertac-core) generates all the log entries. The format is also designed to be easy to process by re-creating copies of the original objects, and calling their methods. This allows the game-replay behavior of the visualizer, as well as data extraction from the re-created objects.

The analyzers in powertac-tools/logtool-examples all operate by reading a state log file and extracting data from the objects created as the file is processed. Most of them produce tables in csv format for ease in further analysis via spreadsheet, R, or Python. The format of the tables is described in the header comments for each analyzer.

Analyzers are not difficult to write; in general, there's a setup section that processes the command line and initializes data collectors, methods that respond to events such as message arrivals, per-timeslot reporting, and summary reporting. Some understanding of simulator operation and message structure is needed in most cases. For example. wholesale market interactions for a given timeslot typically take place in the 24 timeslots preceding the delivery of energy.

To use these analyzers, you will need a Java 8 or newer SDK (not just the runtime), and a working installation of maven 3.3 or newer. You will need to clone the powertac-tools repo (git clone https://github.com/powertac/powertac-tools), then run a build in powertac-tools/logtool-examples (mvn clean test).

Tournament processing

In many cases, interesting analyses must be extended across all the games in a tournament or experiment. The TournamentLogtoolProcessor in powertac-tools/python-scripts serves this purpose. It reads the csv file produced by the tournament or experiment manager, and iterates through the games. If the logs for a game are not yet stored locally, it downloads and unpacks them. For each game, it then runs the requested logtool analyzer.

To use TournamentLogtoolProcess.py, you need a working installation of Python 3.4 or newer. You can run it directly from the command line if your operating system support this behavior (Linux and Mac OS should) and if you make the file executable (chmod +x TournamentLogtoolProcessor.py). If you run it without arguments, it will tell you what it needs. If you operating system does not support this behavior (anyone care to test on Windows and tell us?), then you need to be sure that the python command runs python version 3 (it tells you when it starts up, and you can quit by typing quit() at the prompt). If not, then try python3. You can then run it as python3 TournamentLogtoolPocessor.py to get the usage instructions. Alternatively, you can load it into a python IDE (like Idle) and run the iterate() method. As it runs, it will print out lines of the form

{'path': path-to-data-file, 'gameId': game-number}

Each line can be read into python as a dict structure, which gives the game number and the path to the file.