Difference between revisions of "Getting Started"

From powerwiki
Jump to: navigation, search
(Sample Broker)
 
(38 intermediate revisions by the same user not shown)
Line 1: Line 1:
 
This page is intended to help agent developers get started with the Power TAC server and agent framework. Please feel free to add your experiences and insights to the Discussion page (note that you may need to log in to edit that page).
 
This page is intended to help agent developers get started with the Power TAC server and agent framework. Please feel free to add your experiences and insights to the Discussion page (note that you may need to log in to edit that page).
  
Before you get started with the software, you should be familiar with the [https://papers.ssrn.com/sol3/papers2.cfm?abstract_id=2916182 game specification], and you should join the [http://power-tac-developers.975333.n3.nabble.com/ developer's mailing list on nabble].
+
Before you get started with the software, you should be familiar with the [https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3564107 2020 game specification], and you should join the [https://groups.google.com/g/power-tac-support developer's support forum] that replaces the defunct Nabble forum.
  
There are currently two main components needed by broker developers: the simulation server and the sources for the sample broker. In order to use them, you will need a [http://www.oracle.com/technetwork/java/javase/downloads/index.html Java development environment] (version 1.8 or higher), and [http://maven.apache.org/ Apache Maven] (version 3.3.0 or higher). Mac users may want to refer to the [[MacSetup]] page. Release notes and download links for released versions are available on the [[Release History]] page. Note that these components are not packaged as "apps" for any particular operating system, and currently the only user interface is the web-based Visualizer in the server. If someone knows how to package (and automate the packaging) for systems such as Windows or Mac, please feel free to volunteer.
+
==Components==
  
In addition to the Java and Maven tools, you will also need accurate network time synchronization on any machine that will run a broker against a remote server (such as the tournament servers). This requires a working NTP daemon. NTP setup is easy on Linux and Mac machines, but it can be less easy on Windows, especially if you do not have full administrative access. It is not enough to simply have your system clock synchronized when the machine boots; time offsets of over 50 msec can lead to messages being ignored, and clock drift on a heavily loaded machine during a 2-hour simulation session can cause your broker to be disconnected. One popular tool for NTP synchronization on Windows is the [http://www.satsignal.eu/ntp/setup.html Meinberg package].
+
There are currently two main components needed by broker developers: the simulation server and the sources for the sample broker. In order to use them, you will need a [http://www.oracle.com/technetwork/java/javase/downloads/index.html Java development environment] (version 1.11), and [http://maven.apache.org/ Apache Maven] (version 3.6.3 or higher). Mac users may want to refer to the [[MacSetup]] page. Release notes and download links for released versions are available on the [[Release History]] page. Note that these components are not packaged as "apps" for any particular operating system, and currently the only user interface is the web-based Visualizer in the server. If someone knows how to package (and automate the packaging) for systems such as Windows or Mac, please feel free to volunteer.
  
The '''simulation server''' is a Spring application distributed as a maven pom.xml file, along with a sample configuration file and instructions for setting up and running the server. When you run it, Maven will pull in all the server components and dependencies from the Maven Central repository. You may download [https://github.com/powertac/server-distribution/releases/tag/v1.4.1 version 1.4.1], used in the 2017 competition, or a reasonably stable [https://github.com/powertac/server-distribution/archive/master.zip development snapshot].
+
To maximize performance, communication between a broker and a simulation server is done using asynchronous messaging. In addition to the Java and Maven tools, you will need accurate network time synchronization on any machine that will run a broker against a remote server (such as the tournament servers). This requires a working [https://en.wikipedia.org/wiki/Network_Time_Protocol NTP] daemon. NTP setup is easy on Linux and Mac machines, but it can be less easy on Windows, especially if you do not have full administrative access. It is not enough to simply have your system clock synchronized when the machine boots; time offsets of over 50 msec can lead to messages being ignored, and clock drift on a heavily loaded machine during a 2-hour simulation session can cause your broker to be disconnected. One popular tool for NTP synchronization on Windows is the [http://www.satsignal.eu/ntp/setup.html Meinberg package].
  
The '''sample broker''' is distributed in source form, along with instructions for building and running the broker. The Java version is a Spring application. In older versions, it was packaged as a separate core module and a set of decision services. This allows you to take advantage of newer versions simply by updating the maven configuration. The latest [https://github.com/powertac/sample-broker/releases/tag/v1.4.1 2017 release] supports a number of improvements including the new capacity transaction, and an accounting of total production and consumption in each timeslot. It also removes the broker core to a separate module, which should make it much easier to adapt existing brokers to new features. As always, the [https://github.com/powertac/sample-broker/archive/master.zip development snapshot] is always the most up-to-date. We aim to keep it reasonably stable for the sake of broker developers.
+
The '''simulation server''' is a Spring application distributed as a maven pom.xml file, along with a sample configuration file and instructions for setting up and running the server. When you run it, Maven will pull in all the server components and dependencies from the Maven Central repository. You may download [https://github.com/powertac/server-distribution/releases the latest released version] or a reasonably stable [https://github.com/powertac/server-distribution/archive/master.zip development snapshot].
 +
 
 +
The '''sample broker''' is distributed in source form, along with instructions for building and running the broker. The Java version is a Spring application. In older versions, it was packaged as a separate core module and a set of decision services. This allows you to take advantage of newer versions simply by updating the maven configuration. The latest [https://github.com/powertac/sample-broker/releases release] supports a number of improvements including the new capacity transaction, and an accounting of total production and consumption in each timeslot. It also removes the broker core to a separate module, resolved by Maven, which should make it much easier to adapt existing brokers to new features. As always, the [https://github.com/powertac/sample-broker/archive/master.zip development snapshot] is always the most up-to-date. We aim to keep it reasonably stable for the sake of broker developers.
  
 
If you choose to use the development snapshots, note that the sample broker on github is the HEAD of the development branch, while the server module will pull in the latest deployed snapshot, which is usually more stable than the tip of the development branches of all the components.
 
If you choose to use the development snapshots, note that the sample broker on github is the HEAD of the development branch, while the server module will pull in the latest deployed snapshot, which is usually more stable than the tip of the development branches of all the components.
Line 19: Line 21:
 
= Sample Broker =
 
= Sample Broker =
  
At the time of this writing, the only available implementation of a Power TAC sample broker is in Java. Others, possibly including Repast, R, and/or Python frameworks, are on the agenda but awaiting a volunteer to take responsibility.
+
As of late 2020, the only available implementation of a Power TAC sample broker is in Java, and the core communication and timekeeping framework will likely remain in Java. Others, possibly including a Python framework, are on the agenda but awaiting a volunteer to take responsibility. On the other hand, multiple teams have implemented an out-of-process connection to Python, using either a database or FIFOs between Java and Python.
  
[https://tac-ci.cs.umn.edu/job/sample-broker/ws/target/site/apidocs/index.html Documentation for the Java version is available on the Minnesota Jenkins CI site]. [https://tac-ci.cs.umn.edu/job/powertac-core/ws/target/site/apidocs/index.html Documentation for the powertac-core module is also on the Jenkins CI server]. This is where you will find details on all the message types that your broker can send and receive.
+
[https://powertac.github.io/broker/master/apidocs/ Documentation for the Java version is available on github]. This is where you will find details on the broker core API along with all the message types that your broker can send and receive.
  
 
Broker developers have been using Windows, Mac, and various flavors of Linux successfully. The primary requirements are:
 
Broker developers have been using Windows, Mac, and various flavors of Linux successfully. The primary requirements are:
  
* A Java development kit (JDK) version 1.8 or higher. The Java JRE is not adequate.
+
* A Java development kit (JDK) version 1.11 or higher. The Java JRE is not adequate.
* [http://maven.apache.org/ Apache Maven] version 3.3.0 or higher. Maven provides a rich dependency-management infrastructure as well as goal-directed build automation. If you do not have a copy of maven 3.3.0 or higher on your system, you will need to install it.
+
* [http://maven.apache.org/ Apache Maven] version 3.6.3 or higher. Maven provides a rich dependency-management infrastructure as well as goal-directed build automation. If you do not have a copy of maven 3.6.3 or higher on your system, you will need to install it.
* At least 4 Gb of memory is needed to run both the simulation server and a single broker using an IDE.
+
* At least 4 Gb of memory is needed to run both the simulation server and a single broker using an IDE, and the server can generate log files close to a gigabyte for a standard 9-week simulation session.
* To successfully run a broker against a remote server, both machines must be time-synchronized using ntp, the network time protocol, as described above. Server configurations of most operating systems run the ntp daemon by default, but many desktop setups do not.
+
* To successfully run a broker against a remote server, both machines must be time-synchronized using ntp, the network time protocol, as described above. Server configurations of most operating systems run the ntp daemon by default, but many desktop setups may not.
  
The sample broker, like the Power TAC simulation server and the tournament manager, is a [http://www.springsource.org/spring-framework Spring] application. It can be run from a command-line using [http://maven.apache.org/ Maven], or it can be run from a development environment like [http://www.springsource.org/sts STS], the Spring Tool Suite. STS is Eclipse with a number of additions specific to Spring. Other IDEs can also be used if you prefer, but STS is what most of the server development team uses, with the result that there is help available within the Power TAC community.
+
The sample broker, like the Power TAC simulation server and the tournament manager, is a [http://www.springsource.org/spring-framework Spring] application. It can be run from a command-line using [http://maven.apache.org/ Maven], or it can be run from a development environment like [https://www.eclipse.org/ Eclipse] or [http://www.springsource.org/sts STS], the Spring Tool Suite. STS is Eclipse with a number of additions specific to Spring, but the system builds and runs in the latest Eclipse without problems. Other IDEs can also be used if you prefer, but STS is what most of the server development team uses, with the result that there is help available within the Power TAC community.
  
 
== Build and run the sample broker ==
 
== Build and run the sample broker ==
Line 40: Line 42:
 
This will download a number of modules, then build the sample broker and run a few JUnit tests. If this does not work, then you most likely have a problem with either your Java or Maven setup.
 
This will download a number of modules, then build the sample broker and run a few JUnit tests. If this does not work, then you most likely have a problem with either your Java or Maven setup.
  
The next step is to load the sample broker code into your IDE, then make sure you can build and run it there. In STS, you do this with <code>Import->Existing Maven project</code>, then choose the directory where you unpacked the sample broker code. You should now be able to build and test the agent code with your IDE.
+
The next step is to load the sample broker code into your IDE, then make sure you can build and run it there. In Eclipse, you do this with <code>Import->Existing Maven project</code>, then choose the directory where you unpacked the sample broker code. You should now be able to build and test the agent code with your IDE.
  
 
When you are ready to run your broker, details on the command-line arguments are given in the [https://github.com/powertac/sample-broker/blob/master/README.md README file] contained in the source distribution. You can, of course, use those same arguments inside your IDE by setting up "run configurations". This will allow you to set breakpoints and debug your broker. If you want to use breakpoints, you may want to make the server pause and wait when the broker hits a breakpoint. This can be done by (1) adding the property <code>server.brokerPauseAllowed = true</code> to the server configuration file in server-distribution/config/server.properties, and (2) using the command-line argument <code>--interactive</code> when starting the broker.
 
When you are ready to run your broker, details on the command-line arguments are given in the [https://github.com/powertac/sample-broker/blob/master/README.md README file] contained in the source distribution. You can, of course, use those same arguments inside your IDE by setting up "run configurations". This will allow you to set breakpoints and debug your broker. If you want to use breakpoints, you may want to make the server pause and wait when the broker hits a breakpoint. This can be done by (1) adding the property <code>server.brokerPauseAllowed = true</code> to the server configuration file in server-distribution/config/server.properties, and (2) using the command-line argument <code>--interactive</code> when starting the broker.
Line 48: Line 50:
 
== Modify the sample broker ==
 
== Modify the sample broker ==
  
The sample broker ([http://tac-ci.cs.umn.edu:8080/job/sample-broker/ws/target/site/apidocs/index.html javadoc]) is a Spring application based on three major modules.
+
The sample broker ([https://powertac.github.io/broker/master/apidocs/ javadoc]) is a Spring application based on three major modules.
# The '''common''' module in <code>org.powertac.common</code> is shared with the simulation server, and contains the elements of the Power TAC domain model along with some basic shared infrastructure such as repositories, xml-serialization code, and a configuration engine based on the Apache Commons configuration library. Source is available in the [https://github.com/powertac/common common repo on github].
+
# The '''common''' module in the <code>org.powertac.common</code> package and sub-packages is shared with the simulation server, and contains the elements of the Power TAC domain model along with some basic shared infrastructure such as repositories, xml serialization code, and a configuration engine based on the Apache Commons configuration library. Source is available in the [https://github.com/powertac/powertac-core powertac-core repo on github].
# The '''broker core''' module in <code>org.powertac.samplebroker.core</code> contains the broker framework and the interface to the simulation server, including the broker-server communication protocol. Source is available in the [https://github.com/powertac/broker-core broker-core repo on github].
+
# The '''broker core''' module in <code>org.powertac.samplebroker.core</code> contains the broker framework and the interface to the simulation server, including the broker-server communication protocol. Source is included in the powertac-core repo.
 
# A set of '''example behaviors''' implemented as Spring service classes in <code>org.powertac.samplebroker</code>, including the ContextManager, the PortfolioManager, and the MarketManager. These implement a basic set of agent behaviors, driven primarily by incoming message traffic from the server. They are intended to demonstrate recommended processes for handling messages, for per-timeslot activation, and for initialization and configuration. You are welcome to modify them or replace them with your own designs. Note that message handling is done with overloaded <code>handleMessage()</code> methods that are dispatched from the core module using Java reflection, and configuration is based on an annotation scheme.  
 
# A set of '''example behaviors''' implemented as Spring service classes in <code>org.powertac.samplebroker</code>, including the ContextManager, the PortfolioManager, and the MarketManager. These implement a basic set of agent behaviors, driven primarily by incoming message traffic from the server. They are intended to demonstrate recommended processes for handling messages, for per-timeslot activation, and for initialization and configuration. You are welcome to modify them or replace them with your own designs. Note that message handling is done with overloaded <code>handleMessage()</code> methods that are dispatched from the core module using Java reflection, and configuration is based on an annotation scheme.  
  
 
The recommended way to build your broker is to modify or replace the source code in the <code>org.powertac.samplebroker</code> package, or to use this code as inspiration for your own code in your own packages. Although it is not absolutely necessary, we recommend that you build your broker as a maven project, or using some other tool that uses maven repositories, such as Gradle. This will eliminate the necessity to keep track of all the dependencies on Spring and Apache components.
 
The recommended way to build your broker is to modify or replace the source code in the <code>org.powertac.samplebroker</code> package, or to use this code as inspiration for your own code in your own packages. Although it is not absolutely necessary, we recommend that you build your broker as a maven project, or using some other tool that uses maven repositories, such as Gradle. This will eliminate the necessity to keep track of all the dependencies on Spring and Apache components.
The <code>org.powertac.common</code> and <code>org.powertac.samplebroker.core</code> modules will be retrieved as needed by Maven. You can download sources from github for the [https://github.com/powertac/common common] or [https://github.com/powertac/broker-core broker core] modules and import them into your IDE. Release downloads are available under the "Releases" tab, and the current development snapshot is available either by cloning or downloading a zip file near the lower-right corner of the page.
+
The <code>org.powertac.common</code> and <code>org.powertac.samplebroker.core</code> modules will be retrieved as needed by Maven. You can download sources from github for the [https://github.com/powertac/powertac-core powertac-core] modules and import them into your IDE (make sure you import them as maven projects, otherwise you will not see their dependencies. Release downloads are available under the "Releases" tab, and the current development snapshot is available either by cloning or downloading a zip file near the upper-right corner of the page.
  
The broker agent is configured by adding name=value pairs to <code>src/main/resources/config/broker.properties</code>. You can also specify a separate configuration file using the <code>-config</code> option when you start the broker. Each property name is composed of the classname (without the org.powertac prefix) followed by the attribute name of a configurable value. Configurable values are instance variables or setter methods annotated in the source with <code>@ConfigurableValue</code> along with type and a description. Configuration happens when a service calls <code>propertiesService.configureMe(this)</code>. See the configurable values and the <code>init()</code> method in PortfolioManager for some examples. If you want to configure classes that are not in org.powertac, you can simply use the entire qualified classname.
+
The broker agent is configured by adding name=value pairs to <code>src/main/resources/config/broker.properties</code>. You can also specify a separate configuration file using the <code>-config</code> option when you start the broker. Each property name is composed of the classname (without the <code>org.powertac</code> prefix) followed by the attribute name of a configurable value. Configurable values are instance variables or setter methods annotated in the source with <code>@ConfigurableValue</code> along with type and a description. Configuration happens when a service calls <code>propertiesService.configureMe(this)</code>. See the configurable values and the <code>init()</code> method in PortfolioManager for some examples. If you want to configure classes that are not in org.powertac, you can simply use the entire qualified classname.
  
All interaction with the simulation server is through xml-serialized jms messages. You can write code to receive them by writing a <code>handleMessage(msg)</code> method to process each message type you want to see. All messages are instances of classes in packages org.powertac.common and org.powertac.common.msg in the [https://github.com/powertac/common common module]. You may, of course, add additional message-handling modules simply by following the pattern in the existing MarketManagerService and PortfolioManagerService (see [https://github.com/powertac/sample-broker/blob/master/src/main/java/org/powertac/samplebroker/MarketManagerService.java MarketManagerService.java] for example). Any module that implements the <code>Initializable</code> interface will have its <code>initialize(BrokerContext context)</code> method called before each simulation session, and all the <code>handleMessage()</code> methods will be registered to receive their respective message types. Outgoing messages are sent to the server by calling the <code>sendMessage(msg)</code> method on the context. [http://tac-ci.cs.umn.edu:8080/job/sample-broker/ws/target/site/apidocs/index.html Javadocs are posted] on our Jenkins CI server.
+
All interaction with the simulation server is through xml-serialized jms messages. You can write code to receive them by writing a <code>handleMessage(msg)</code> method to process each message type you want to see. All messages are instances of classes in packages <code>org.powertac.common</code> and <code>org.powertac.common.msg</code> in the common submodule of [https://github.com/powertac/powertac-core powertac-core]. You may, of course, add additional message-handling modules simply by following the pattern in the existing MarketManagerService and PortfolioManagerService (see [https://github.com/powertac/sample-broker/blob/master/src/main/java/org/powertac/samplebroker/MarketManagerService.java MarketManagerService.java] for an example). Any module that implements the <code>Initializable</code> interface will have its <code>initialize(BrokerContext context)</code> method called before each simulation session, and all the <code>handleMessage()</code> methods will be registered to receive their respective message types. Outgoing messages are sent to the server by calling the <code>sendMessage(msg)</code> method on the context.
  
The server sends its per-timeslot messages out in batches, and the last one is guaranteed to be a TimeslotComplete message. When that message arrives, all modules that implement the <code>Activatable</code> interface will have their <code>activate()</code> methods called, in no particular order. If anyone feels it is important to control the order of activation, it would require a relatively easy modification in the core, along with some additional configuration. You may request such a change, report defects, or recommend other improvements, through the [https://github.com/powertac/powertac-server/issues Power TAC issue tracker].
+
The server sends its per-timeslot messages out in batches, and the last one is guaranteed to be a <code>TimeslotComplete</code> message. When that message arrives, all modules that implement the <code>Activatable</code> interface will have their <code>activate()</code> methods called, in no particular order. If anyone feels it is important to control the order of activation, it would require a relatively easy modification in the core, along with some additional configuration. You may request such a change, report defects, or recommend other improvements, through the [https://github.com/powertac/powertac-server/issues Power TAC issue tracker].
  
Brokers also need to interact with the Power TAC Tournament Manager, through a REST interface. This interaction is encapsulated in the <code>core.BrokerTournamentService</code> module. The only purpose of this interaction is to acquire credentials and access details for simulation servers that run under control of the Tournament Manager. You should not need to modify this code.
+
Brokers also need to interact with the Power TAC Tournament Manager, through a REST interface. This interaction is encapsulated in the <code>samplebroker.core.BrokerTournamentService</code> class in the broker-core submodule. The only purpose of this interaction is to acquire credentials and access details for simulation servers that run under control of the Tournament Manager. You should not need to modify this code.
  
 
If you want to add a new top-level module to the broker, it should be enough to simply annotate it with <code>@Service</code> ahead of the class declaration, implement the <code>Initializable</code> interface (provides per-session module initialization), and possibly implement the <code>Activatable</code> interface (provides per-timeslot activation). MarketManagerService and PortfolioManagerService are annotated this way. This is all the information Spring needs to instantiate it and wire it in through dependency injection, as long as the module is in a package Spring knows about. That bit of configuration is in <code>src/main/resources/broker.xml</code>. As shipped, all Service modules in <code>org.powertac</code> or any subpackage are included by Spring. If you want to use your own packages, you simply need to add another component-scan clause listing the package name at the top of your own package hierarchy.
 
If you want to add a new top-level module to the broker, it should be enough to simply annotate it with <code>@Service</code> ahead of the class declaration, implement the <code>Initializable</code> interface (provides per-session module initialization), and possibly implement the <code>Activatable</code> interface (provides per-timeslot activation). MarketManagerService and PortfolioManagerService are annotated this way. This is all the information Spring needs to instantiate it and wire it in through dependency injection, as long as the module is in a package Spring knows about. That bit of configuration is in <code>src/main/resources/broker.xml</code>. As shipped, all Service modules in <code>org.powertac</code> or any subpackage are included by Spring. If you want to use your own packages, you simply need to add another component-scan clause listing the package name at the top of your own package hierarchy.
  
The sample code synchronizes access to variables between the <code>handleMessage()</code> methods and the <code>activate()</code> method in the service modules. This is necessary because the <code>handleMessage()</code> methods run in threads managed by the JMS service, while the <code>activate()</code> methods run in a thread that is driven by the simulation clock.  
+
The sample code synchronizes access to variables between the <code>handleMessage()</code> methods and the <code>activate()</code> method in the service modules. This is necessary because the <code>handleMessage()</code> methods run in threads managed by the JMS service, while the <code>activate()</code> methods run in a thread driven by the simulation clock. Failure to synchronize access to data that is shared by <code>handleMessage()</code> and <code>activate()</code> methods can result in weird errors.
 +
 
 +
In a tournament situation, Broker agents are commonly started once for a series of games, which means that you have to be careful about how they re-initialize between games. Modules that implement the <code>Initializable</code> interface will have their <code>initialize()</code> methods called before the agent attempts to log in for the next game. All initialization code should be in <code>initialize()</code> methods, and not in a constructor or elsewhere, otherwise your agent may not recover correctly between games. It is important that your initialization code clears out all per-game data. Note that the ID counter is re-initialized at this time, so if you fail to remove instances that use automatically-generated ID values, you can encounter duplicates, another possible source of weird errors.
 +
 
 +
When a broker logs in to the simulation server, it receives in the "broker-accept" message a number that's used as a prefix for the ID values in objects created and sent to the server during a game. This is important, because the server uses those object ID values as keys in various maps. A consequence is that it does not work to keep objects (like tariff specifications, for example) over between games - if your ID prefix is different in the next game, and it often is, your object IDs can collide with objects from some other broker. The server will avoid this by rejecting these messages, so you need to either re-create those objects, or update their IDs at the beginning of each game. You can get new ID values using <code>IdGenerator.createId()</code> after the login is complete. A good place to do that would be in <code>ContextManagerService.handleMessage(Competition)</code>.
 +
 
 +
== Interpret and compose messages ==
 +
 
 +
Once our broker is logged in, all the interaction between the simulation server and your broker is through asynchronous messages. Some message types are sent by the broker to the server, some sent by the server to the broker. A few, like <code>TariffSpecification</code>, are sent in both directions. All of them are in either the <code>org.powertac.common</code> or the <code>org.powertac.common.msg</code> packages. See [https://powertac.github.io/broker/master/apidocs/ the javadocs] for full details, and contribute to the discussion on this page if you have ideas on how we could improve our documentation. It's always a work in progress, and our time and resources are limited.
 +
 
 +
=== At login ===
 +
 
 +
In a development environment, the broker simply logs into the server, and the server ignores the password. In a tournament environment, the tournament manager supplies the password when a broker is assigned to a particular server, and each server is supplied with a list of assigned brokers at the start of each session.
 +
 
 +
;<code>BrokerAuthentication</code>
 +
: Passes broker username and password to the server at the start of a session.
 +
 
 +
;<code>BrokerAccept</code>
 +
: Passed from server to broker to indicate broker has successfully logged in. Contains three fields:
 +
:<code>prefix</code> is used to see the ID generator to ensure that message ID values will be unique among brokers.
 +
:<code>key</code> is a short message prefix used to prevent spoofing and unintended broker holdovers between successive games. This will be inserted at the front of each message sent to the server.
 +
:<code>serverTime</code> is the server's local time when the <code>BrokerAccept</code> message is generated. This can be used to estimate network latency, and to ensure that the broker's local time is not out of range.
 +
 
 +
=== Start of game ===
 +
 
 +
Once a broker has successfully logged in, the server sends details on the game that can be used to adjust parameters and make initial decisions about pricing and other market activities.
 +
 
 +
;<code>Competition</code>
 +
: Contains information about the server and the game about to start, including the list of competing brokers, the list of customer models, the <code>PomId</code> or version of the server, clock parameters, timeslot parameters, parameters for wholesale and balancing markets, clock parameters. See [https://powertac.github.io/broker/master/apidocs/org/powertac/common/Competition.html the javadoc] for details.
 +
 
 +
;<code>CustomerInfo</code>
 +
: Inside the <code>Competition</code> message is a list of <code>CustomerInfo</code> structures. Each gives information about one of the customer models, including its population, its <code>PowerType</code> (whether it's a producer, consumer or storage type, whether it's curtailable, and how much regulation capacity it can offer.
 +
 
 +
;<code>CustomerBootstrapData</code>
 +
: For each customer, gives the actual consumption/production for each timeslot of the bootstrap period.
 +
 
 +
;<code>MarketBootstrapData</code>
 +
: Gives the amount of energy traded and prices for the bootstrap period.
  
In a tournament situation, Broker agents are commonly started once for a series of games, which means that you have to be careful about how they re-initialize between games. Modules that implement the <code>Initializable</code> interface will have their <code>initialize()</code> methods called before the agent attempts to log in for the next game. All initialization code should be in <code>initialize()</code> methods, and not in a constructor or elsewhere, otherwise your agent may not recover correctly between games.
+
;<code>SimStart</code>
 +
: Signals the start of the simulation session, communicates the server time at the start to allow the broker to synchronize its own simulation clock.
  
When a broker logs in to the simulation server, it receives in the "broker-accept" message a number that's used as a prefix for the ID values in objects created and sent to the server during a game. This is important, because the server uses those object id values as keys in various maps. A consequence is that it does not work to keep objects (like tariff specifications, for example) over between games - if your id prefix is different in the next game, and it often is, your object ids can collide with objects from some other broker. The server will avoid this by rejecting these messages, so you need to either re-create those objects, or update their ids at the beginning of each game. You can get new id values using <code>IdGenerator.createId()</code> after the login is complete. A good place to do that would be in <code>ContextManagerService.handleMessage(Competition)</code>.
+
=== Timeslots and timekeeping ===
 +
Messages include start and end markers for each timeslot, and information about pauses, which update clock parameters.
  
= Power TAC simulation server =
+
;<code>TimeslotUpdate</code>
 +
: Sent at the start of each timeslot, gives the first and last timeslot for which the wholesale market will accept new orders during the current timeslot.
 +
 
 +
;<code>TimeslotComplete</code>
 +
: This is always the last message from the server for the current timeslot.
 +
 
 +
;<code>PauseRequest</code>
 +
: If the server is configured to allow broker-initiated pauses, then this is the message sent by a broker to request a pause. If it succeeds, the server will respond with a <code>SimPause</code> message to all brokers.
 +
 
 +
;<code>PauseRelease</code>
 +
: If the server is currently paused by the broker sending this message, then the pause will be released and a <code>SimResume</code> message will be sent to brokers.
 +
 
 +
;<code>SimPause</code>
 +
: Sent by the server to indicate that the simulation clock is paused. This can happen if a broker requests a pause, and it can happen if the server takes too much time processing a single timeslot. This is sometimes necessary to guarantee that brokers have a minimum amount of time to make their decisions in each timeslot. The server property <code>server.simulationClockControl.minAgentWindow</code> sets this minimum.
 +
 
 +
;<code>SimResume</code>
 +
: Sent by the server to indicate that a pause is released, and gives updated clock parameters to re-synchronize server and broker clocks.
 +
 
 +
=== Tariff market ===
 +
 
 +
These messages are the interface between brokers and the tariff market, and between brokers and their customers. New tariffs are normally published every six hours, at which time a fee must be paid for publication. Customers can evaluate tariffs at any time, but typically do so no more often than every six hours. Most customers ignore new tariff publications most of the time as described in the game specification.
 +
 
 +
;<code>TariffSpecification</code>
 +
: This is a description of a new tariff offering. It is sent from a broker to the server to publish a new tariff, and sent from the server to brokers when tariffs by other brokers are published. Properties include <code>powerType</code> to specify which types of customers may subscribe, a <code>signupPayment</code> to reward (positive value) or punish (negative value) a customer for subscribing, a <code>periodicPayment</code> that the customer must pay daily regardless of energy production or consumption, a <code>minDuration</code> that specifies the minimum time a customer must stay subscribed to avoid paying an <code>earlyWithdrawPayment</code> (negative if the customer pays the broker), and potentially the ID of an existing tariff that <code>supersedes</code> this tariff. Note that the supersede relationship only applies if the superseded tariff is revoked. If <code>expiration</code> is specified, it is the time after which new subscriptions to this tariff will not be accepted. It is specified as a <code>long</code> giving the number of milliseconds since 1970-01-01T00:00:00Z, which can be computed by retrieving the current time from the <code>TimeService</code>, calling <code>getMillis()</code> on the returned <code>Instant</code> value, and adding the desired number of hours converted to msec. A valid tariff must include at least one <code>Rate</code>, and zero or one <code>RegulationRate</code>.
 +
 
 +
;<code>Rate</code>
 +
: A <code>Rate</code> specifies the cost of power potentially over a specified time interval and under specific conditions of consumption or production. A <code>fixed</code> rate specifies a single value, which is the cost (negative value) to the customer for consuming a unit (one kWh) of energy. If the rate is not fixed, then it must specify a <code>minValue</code>, a <code>maxValue</code>, and an <code>expectedMean</code> value. The tariff market keeps track of the actual realized prices for variable-rate tariffs, so customers can use this information in their evaluations. A variable-rate must also specify a <code>noticeInterval</code>, which is the minimum horizon over which subscribers must be notified with <code>VariableRateUpdate</code> messages containing <code>HourlyCharge</code> instances.
 +
: A <code>Rate</code> can also specify when it is applicable; the <code>dailyBegin</code> and <code>dailyEnd</code> fields give hours of the day (0 .. 23) when the rate comes into effect and goes out of effect, and the <code>weeklyBegin</code> and <code>weeklyEnd</code> fields specify the days when they apply (1 .. 7). So a rate with <code>dailyBegin</code> = 7, <code>dailyEnd</code> = 12, <code>weeklyBegin</code> = 6, and <code>weeklyEnd</code> = 1 applies from 7:00 until noon on Saturdays and Sundays. If these time-of-use rates are included, the set of rates must cover all times. So if we have a rate starting at 6 and ending at noon, one starting at noon and ending at 17, one starting at 17 and ending at 23, and one starting at 23 and ending at 6, we have covered all hours. If weekly start and end days are also included, they must also cover the week.
 +
: Finally, <code>Rate</code>s can be used to specify "tiered" or "block" pricing that charge more or less for consumption/production over some threshold in a day. So if we have a rate with a fixed <code>value</code> = .10 and <code>tierThreshold</code> = 0 and another with a fixed <code>value</code> = .15 and <code>tierThreshold</code> = 10, then the first 10 kWh during a given day will be billed at .10/kWh, and the remainder will be billed at .15/kWh.
 +
 
 +
;<code>RegulationRate</code>
 +
: Used to specify per-kWh payments (positive values) to the customer for up-regulation (which may be achieved by reducing demand) and charges (negative values) for down-regulation. These prices become bids in the balancing market in every timeslot, and can be exercised to the extent that customers report their regulation capacity. The <code>response</code> field is ignored in version 1.8 and earlier, and there are currently no plans to use it.
 +
 
 +
;<code>TariffStatus</code>
 +
: Sent to a broker to report the success or failure of new tariff publication, and of tariff updates such as <code>TariffRevoke</code> and <code>TariffExpire</code>. If the status value is not <code>Status.success</code>, then the new tariff or update has failed, and the status value indicates the reason for failure.
 +
 
 +
;<code>TariffTransaction</code>
 +
: Sent to brokers to report on credits or debits related to specific tariffs. The most common transaction types are <code>PRODUCE</code> and <code>CONSUME</code>. For a tariff with periodic payments, a <code>PERIODIC</code> transaction is issued to represent the fixed payment. These transactions are issued by each subscribed customer in each timeslot, and for population models they also include a <code>customerCount</code> value to indicate the number of individual customers that are contributing to the transaction. Other transaction types include <code>PUBLISH</code>, indicating that a tariff has been published, <code>SIGNUP</code> when customers subscribe to the tariff, <code>WITHDRAW</code> when customers withdraw from the tariff, <code>REVOKE</code> to indicate that a tariff has been successfully revoked, and <code>REFUND</code> when a customer withdraws from a tariff that included a negative signup payment.
 +
 
 +
;<code>VariableRateUpdate</code>
 +
: If a <code>Rate</code> specifies <code>fixed</code> = false, then it is a "dynamic-price" rate, and the customer must be informed of the future prices that will be charged over the promised <code>noticeInterval</code>. To accomplish this, the broker must issue a <code>VariableRateUpdate</code> containing an <code>HourlyCharge</code> for each hour in the future over the notice interval. So if the notice interval is 6 hours, then 6 <code>VariableRateUpdate</code> messages must be sent immediately to cover the first six hours, and one additional <code>VariableRateUpdate</code> must be sent in each subsequent hour. Some customer models will use this information to optimize their usage.
 +
 
 +
;<code>HourlyCharge</code>
 +
: Specifies the per-kWh charge for a <code>VariableRateUpdate</code>.
 +
 
 +
;<code>EconomicControlEvent</code>
 +
: Allows a broker to request curtailment for a specific future timeslot from a subscribed <code>INTERRUPTIBLE_CONSUMPTION</code> or <code>STORAGE</code> customer. In addition to a tariff, two fields must be specified: <code>timeslotIndex</code> specifies the timeslot when this control is to be applied, and <code>curtailmentRatio</code> specifies the amount of curtailment to be applied, as a ratio of the total flexibility currently available. The broker can use this to reduce (0 < ratio <= 1) consumption, withdraw energy from storage (1 < ratio <= 2), or increase (0 > ratio >= -1) consumption at times of extreme wholesale prices, or when it expects to be imbalanced, but of course the broker will not know the actual impact of its order because it does not know precisely how much flexibility a given customer will have in the current or future timeslots. Of course, repeated use of ratios > 0 will quickly deplete up-regulation capacity, and repeated use of ratios < 0 will quickly deplete down-regulation capacity of the customers subscribed to a given tariff.
 +
 
 +
;<code>TariffExpire</code>
 +
: Changes the expiration time of the specified tariff, after which new subscriptions will not be allowed.
 +
 
 +
;<code>TariffRevoke</code>
 +
: Revokes an existing tariff. Assuming that tariff has already been specified as being superseded by a new tariff, then all affected customers will be switched to the superseding tariff, but they are also notified and are free to choose some other tariff instead. In fact, revoking a tariff annoys most customers, and unless the superseding tariff is a better deal than its current competitors, customers may instead choose some other tariff. There is a fee for revoking a tariff, so a <code>TariffTransaction</code> will be issued to notify the broker of the fee.
 +
 
 +
=== Wholesale market ===
 +
 
 +
;<code>Order</code>
 +
: An <code>Order</code> is a bid in the wholesale market. A positive quantity of energy is an offer to buy energy (in MWh), and a negative quantity of money is an offer to pay up to that amount per MWh. A negative energy quantity is an offer to sell, the price being the minimum acceptable payment per MWh. If the price is left empty, then it is a "market order" and will transact at the market clearing price. See Section 5.2 of the game specification for details on how market clearing works.
 +
 
 +
;<code>OrderStatus</code>
 +
: This message is sent to the broker if an order could not be processed, due to the specified timeslot being out of range. So if the current timeslot is <i>n</i> and the wholesale market is configured to trade in timeslots <i>n</i> + 1 through <i>n</i> + 24, then an order for timeslot <i>n</i> would result in the server sending an <code>OrderStatus</code> message.
 +
 
 +
;<code>ClearedTrade</code>
 +
: Conveys information (how much was traded at what time and price, and for which timeslot) about market clearings to all brokers for each timeslot in which energy was traded.
 +
 
 +
;<code>Orderbook</code>
 +
: Conveys information, in the form of a set of <code>OrderbookOrder</code> instances, about uncleared bids and asks for each timeslot in which the market clears. This allows brokers to view unsatisfied demand and supply and to estimate the shapes of the supply and demand curves.
 +
 
 +
;<code>OrderbookOrder</code>
 +
: Represents an uncleared <code>Order</code> from a market clearing, giving the requested quantity and limit price.
 +
 
 +
;<code>MarketTransaction</code>
 +
: Informs brokers about their own cleared market transactions, giving quantity traded, price, and delivery timeslot. The actual money and energy will trade hands at the target timeslot, not at the timeslot when the order(s) cleared.
 +
 
 +
;<code>MarketPosition</code>
 +
: Sent at the conclusion of trading in each timeslot to inform the broker of the total amount of energy that will be transfered in a specific future timeslot for which one or more of the broker's <code>Order</code>s have cleared.
 +
 
 +
=== Balancing market ===
 +
 
 +
;<code>BalanceReport</code>
 +
: Reports the total net imbalance for the current timeslot. Brokers who carry an imbalance with the opposite sign (and a smaller magnitude) stand to gain when the balancing market clears.
 +
 
 +
;<code>BalancingControlEvent</code>
 +
: Reports the amount of regulation applied to customers of a given tariff that results from the balancing process. The tariff must be for a customer of type <code>INTERRUPTIBLE</code> or <code>STORAGE</code>. Regulation can be applied in two cases: (1) the tariff contains a <code>RegulationRate</code> specifying a price that clears the balancing market, or (2) the broker has issued a <code>BalancingOrder</code> against the tariff with a price that clears the market. Note that every <code>BalancingControlEvent</code> will be accompanied by a <code>TariffTransaction</code> giving the amount of energy involved and the market clearing price, which will always be no lower than the price specified in the <code>RegulationRate</code> or <code>BalancingOrder</code>.
 +
 
 +
;<code>BalancingOrder</code>
 +
: An offer of balancing capacity at a specified ratio and price to the Balancing Market. In most cases, these are generated internally when a new <code>TariffSpecification</code> containing a <code>RegulationRate</code> is processed, in which case the ratio is ignored. For interruptible or storage tariffs that lack regulation rates, the <code>BalancingOrder</code> specifies the amount of regulation that can be applied using the <code>ratio</code> field: if 0 < value <= 1, then consumption in the current timeslot can be curtailed by this ratio; if 1 < value <= 2, then a storage type with ability to discharge (like a battery) can be asked to discharge up to its maximum discharge power; if 0 > value >= -1, then a storage type can be asked to consume additional energy up to the difference between its maximum charge rate and the amount it otherwise would have consumed. Note that the ratio value is ignored for storage types subscribed to a tariff with a regulation rate, because in this case the subscribed customers report their available capacity directly to the balancing market.
 +
 
 +
;<code>BalancingTransaction</code>
 +
: Reports the broker's final supply-demand imbalance in the current timeslot, and the total charge or payment for this imbalance.
 +
 
 +
=== Distribution utility ===
 +
 
 +
;<code>DistributionReport</code>
 +
: Reports the total consumption and total production in the current timeslot across all customers.
 +
 
 +
;<code>DistributionTransaction</code>
 +
: The Distribution utility charges brokers for their use of the grid in (potentially) three different ways: (1) flat per-customer fees (also called meter fees) for each subscribed customer; (2) capacity fees for broker contributions to peak demand; and (3) fees for net energy transport. The default configuration of the server does not charge transport fees, and meter fees are the sum of smaller per-customer fees for "small" customers, and higher fees for "large" customers. The <code>DistributionTransaction</code> is issued in each timeslot, giving the number of small meters, the number of large meters, and the energy transport values that are being assessed, along with the total charge.
 +
 
 +
;<code>CapacityTransaction</code>
 +
: A large fraction of the cost of a grid is a function the peak power it must be capable of transporting. For real-world distribution utilities, this cost can be comparable to total energy costs. In Power TAC, peak capacity assessments are done periodically (every two weeks in the tournament configuration). Details on how peaks are measured and assessed are given in Section 7.2 of the game specification. During each assessment, zero or more (typically up to three) peaks are identified and individual broker contributions measured. Each broker who contributed to a peak is sent a <code>CapacityTransaction</code> giving the timeslot in which the peak occurred, the threshold that was exceeded, the broker's contribution to the peak (which can be negative for brokers whose customers produced more energy than they consumed in that timeslot), and the charge (or credit) for the peak.
  
To test your broker, you will need it to interact with the server. The Power TAC simulation server (or just the "server") is a Spring application, designed to be installed and run using [http://maven.apache.org/ Maven]. Note that the server release packages are simply maven configuration files with some documentation; all the actual code will be downloaded by maven the first time you try to run the server.
+
=== Bank ===
  
== Running the server ==
+
;<code>BankTransaction</code>
 +
: Reports interest charges and payments on a broker's bank account. Issued once/day in the standard tournament configuration.
  
The server runs the Power TAC simulation in two modes; detailed instructions are in the README file included with the server distribution package once you have downloaded and unpacked it:
+
;<code>CashPosition</code>
 +
: Reports the broker's total bank balance, issued once/timeslot after all transactions have been processed.
  
* In "bootstrap" mode, the model runs for a period of time with only the default broker, and collects data on customer power usage, market prices, and weather. This data is saved to a file at the end of the bootstrap period. The resulting data file can be used to run multiple sim sessions.
+
=== Weather service ===
* In "sim" mode, the model loads a bootstrap data file and restores the model to its state at the end of the bootstrap period, then allows brokers to log in and sends them the bootstrap data before starting a simulation. The simulation literally starts at the end of the bootstrap period, so for example the first timeslot is not number 0, but the number of timeslots in the bootstrap period. Note that not all of the timeslot-related data is carried over from the bootstrap sim - for example, you cannot see the orderbooks from timeslots that completed in the bootstrap period.
 
  
As described in the README file, you can run the server as a web-based app under the jetty webserver, or as a command-line app without a visual interface. In bootstrap mode, there is not much to see, and it runs too fast for the visualizer to keep up. However, you can run a bootstrap session using the visualizer to generate the bootstrap data file needed by the full simulator.
+
;<code>WeatherReport</code>
 +
: Reports weather conditions for the current timeslot. Fields include <code>temperature</code>, <code>windSpeed</code, <code>windDirection</code>, and <code>cloudCover</code>.
  
=== Using the visualizer (version 1) ===
+
;<code>WeatherForecast</code>
 +
: Contains a list of <code>WeatherForecastPrediction</code> instances, one for each of the following 24 hours.
  
To run the simulator/visualizer combination, you will need a machine with at least 4 Gb of memory and several Gb of free disk space. Two or more cores will be helpful, although the server, visualizer, and sample broker should not be compute-bound on any reasonably hefty desktop or laptop machine. When you start the server using
+
;<code>WeatherForecastPrediction</code>
 +
: Reports predicted weather conditions for a specified future timeslot. Fields are the same as <code>WeatherReport</code>.
  
  mvn -P web
+
=== End of game ===
  
it will (if necessary) download modules from the maven repository, then start an instance of the jetty webserver containing the server and visualizer. Once jetty is running, browse to http://localhost:8080/visualizer to get started. If you navigate to the "Settings" page, you will see a form that allows you to set up and run boot and sim sessions. The top of that page is shown here:
+
;<code>SimEnd</code>
 +
: Sent at the end of a simulation session. This is guaranteed to be the last message sent by the server in a session.
  
[[File:Vcp1.png|center]]
+
= Power TAC simulation server =
  
Common options allow you to select an alternate configuration file, and to specify a suffix for naming log files. If you leave them both blank, it will use the default configuration in <code>config/server.properties</code>, and the logfiles will be named <code>log/powertac-boot-0.trace</code> and <code>log/powertac-boot-0.state</code> for a boot session. Since a standard sim runs for about 2 hours, there is a sample configuration file <code>short-game.props</code> provided that will finish in around 10 minutes. You are welcome to edit these files or copy and modify them as you see fit. Details on the various configuration options and what they do are provided in comments in <code>config/server.properties</code>. Keep in mind that some values (like the competition.timeslotLength) may not have the expected effect if you change them, but values like interest rates and fees should have the expected effects.
+
To test your broker, you will need it to interact with the server. The Power TAC simulation server (or just the "server") is a Spring application, designed to be installed and run using [http://maven.apache.org/ Maven]. Note that the server release packages are simply maven configuration files with some documentation; all the actual code will be downloaded by maven the first time you try to run the server.
  
To run a boot session, scroll down to the boot panel (shown below), fill in the name of the file you wish to have the bootstrap data saved to (for example, <code>boot.xml</code>, and poke the "Run" button. A boot session normally takes less than 5 minutes to complete.
+
Information about modifying and building the server is available on the [https://github.com/powertac/powertac-server/wiki developer's wiki at github].
  
[[File:Boot-mode.png|center]]
+
== Running the server ==
  
Once you have a bootstrap-data file available, you can run sim sessions. Simply scroll down to the sim panel (shown below), fill in the name of a bootstrap-data file (created by a boot session), add brokers, and start the session with the "Run" button. The server will load the bootstrap data, then wait for the brokers to log in before starting the simulation. Brokers are separate programs identified by their "usernames" -- this is the value of the <code>samplebroker.core.powerTacBroker.username</code> property in <code>sample-broker/broker.properties</code>. Note that the broker username is case-sensitive; in other words, the username in the broker.properties file must exactly match the name filled in as the broker name in the visualizer dialog, including capitalization. The username may not contain embedded spaces, and the only acceptable characters are alphanumeric and underscore.
+
The server runs the Power TAC simulation in two modes; detailed instructions are in the README file included with the server distribution package once you have downloaded and unpacked it:
  
[[File:Sim-mode.png|center]]
+
* In "bootstrap" mode, the model runs for a period of time with only the default broker, and collects data on customer power usage, market prices, and weather. This data is saved to a file at the end of the bootstrap period. The resulting data file can be used to run multiple sim sessions.
 +
* In "sim" mode, the model loads a bootstrap data file and restores the model to its state at the end of the bootstrap period, then allows brokers to log in and sends them the bootstrap data before starting a simulation. The simulation literally starts at the end of the bootstrap period, so for example the first timeslot is not number 0, but the number of timeslots in the bootstrap period (typically 360). Note that not all timeslot-related data is carried over from the bootstrap session - for example, you cannot see the orderbooks from timeslots that completed in the bootstrap period.
  
The broker is started with <code>mvn exec:exec</code>. Additional options are explained in the README file.
+
As described in the README file, you can run the server as a web-based app under the jetty webserver, or as a command-line app without a visual interface. In bootstrap mode, there is not much to see, and it typically runs too fast for the visualizer to keep up. However, you can run a bootstrap session using the visualizer to generate the bootstrap data file needed for a full simulation session.
  
If you want to run a broker on a separate machine, you will have to choose a JMS URL that is visible on the network. The default url is <code>tcp://localhost:61616</code>; if you want to use an external URL and you are running the server on a machine "tardis.powertac.org", the url would be <code>tcp://tardis.powertac.org:61616</code>. This value would also have to be set in the broker's broker.properties file for all participating brokers.
+
=== Using the visualizer (version 1) ===
  
Once all the brokers have logged in and the boot data has been loaded, the Game Status indicator at the top of the page will change to "running" and you can poke the Game View link to see the visualizer display. In the visualizer display, the wrench icon brings you back to the competition control page.
+
This version is obsolete and no longer supported.
  
  
Line 120: Line 267:
 
This will start up a server on http://localhost:8080. It takes a little while for it to start up, so please be patient.
 
This will start up a server on http://localhost:8080. It takes a little while for it to start up, so please be patient.
  
Like the version 1 visualizer, it allows you to run boot games, and subsequently -- given the record of a boot game -- to run simulations with brokers. Unlike the version 1 visualizer, you can replay earlier games (either ones that you've run within the visualizer, or using the state logs from external sources). You can even provide an URL to a compressed log bundle as produced during the annual Tournament, e.g. one of the entries at http://ts.powertac.org/log_archive/finals_2016_06/ , and replay that directly.
+
This allows you to run boot sessions, and subsequently -- given the record of a boot session -- to run simulations with brokers. You can replay earlier games (either ones that you've run locally or using the state logs from external sources). You can even provide an URL to a compressed log bundle as produced during the annual Tournament, e.g. one of the entries among the [https://powertac.org/tournament/#section-6 past tournaments on powertac.org], and replay that directly.
  
 
The GUI to set up and start/stop games, of all varieties, is in the Games tab of the application, and ought to be more or less self-explanatory. (You don't see all input fields at once, they become visible as you enter information).
 
The GUI to set up and start/stop games, of all varieties, is in the Games tab of the application, and ought to be more or less self-explanatory. (You don't see all input fields at once, they become visible as you enter information).

Latest revision as of 15:47, 18 September 2021

This page is intended to help agent developers get started with the Power TAC server and agent framework. Please feel free to add your experiences and insights to the Discussion page (note that you may need to log in to edit that page).

Before you get started with the software, you should be familiar with the 2020 game specification, and you should join the developer's support forum that replaces the defunct Nabble forum.

Components

There are currently two main components needed by broker developers: the simulation server and the sources for the sample broker. In order to use them, you will need a Java development environment (version 1.11), and Apache Maven (version 3.6.3 or higher). Mac users may want to refer to the MacSetup page. Release notes and download links for released versions are available on the Release History page. Note that these components are not packaged as "apps" for any particular operating system, and currently the only user interface is the web-based Visualizer in the server. If someone knows how to package (and automate the packaging) for systems such as Windows or Mac, please feel free to volunteer.

To maximize performance, communication between a broker and a simulation server is done using asynchronous messaging. In addition to the Java and Maven tools, you will need accurate network time synchronization on any machine that will run a broker against a remote server (such as the tournament servers). This requires a working NTP daemon. NTP setup is easy on Linux and Mac machines, but it can be less easy on Windows, especially if you do not have full administrative access. It is not enough to simply have your system clock synchronized when the machine boots; time offsets of over 50 msec can lead to messages being ignored, and clock drift on a heavily loaded machine during a 2-hour simulation session can cause your broker to be disconnected. One popular tool for NTP synchronization on Windows is the Meinberg package.

The simulation server is a Spring application distributed as a maven pom.xml file, along with a sample configuration file and instructions for setting up and running the server. When you run it, Maven will pull in all the server components and dependencies from the Maven Central repository. You may download the latest released version or a reasonably stable development snapshot.

The sample broker is distributed in source form, along with instructions for building and running the broker. The Java version is a Spring application. In older versions, it was packaged as a separate core module and a set of decision services. This allows you to take advantage of newer versions simply by updating the maven configuration. The latest release supports a number of improvements including the new capacity transaction, and an accounting of total production and consumption in each timeslot. It also removes the broker core to a separate module, resolved by Maven, which should make it much easier to adapt existing brokers to new features. As always, the development snapshot is always the most up-to-date. We aim to keep it reasonably stable for the sake of broker developers.

If you choose to use the development snapshots, note that the sample broker on github is the HEAD of the development branch, while the server module will pull in the latest deployed snapshot, which is usually more stable than the tip of the development branches of all the components.

If you wish to modify or contribute to further development of the simulation server or the sample broker, you will need to set up a server source environment. Please start by reading through the Getting Started and Development Policies pages on the Power TAC github wiki. You may also want to browse through the Power TAC issue-tracking system on github.

In addition, you may want to download the Power TAC log-analysis tool.

Sample Broker

As of late 2020, the only available implementation of a Power TAC sample broker is in Java, and the core communication and timekeeping framework will likely remain in Java. Others, possibly including a Python framework, are on the agenda but awaiting a volunteer to take responsibility. On the other hand, multiple teams have implemented an out-of-process connection to Python, using either a database or FIFOs between Java and Python.

Documentation for the Java version is available on github. This is where you will find details on the broker core API along with all the message types that your broker can send and receive.

Broker developers have been using Windows, Mac, and various flavors of Linux successfully. The primary requirements are:

  • A Java development kit (JDK) version 1.11 or higher. The Java JRE is not adequate.
  • Apache Maven version 3.6.3 or higher. Maven provides a rich dependency-management infrastructure as well as goal-directed build automation. If you do not have a copy of maven 3.6.3 or higher on your system, you will need to install it.
  • At least 4 Gb of memory is needed to run both the simulation server and a single broker using an IDE, and the server can generate log files close to a gigabyte for a standard 9-week simulation session.
  • To successfully run a broker against a remote server, both machines must be time-synchronized using ntp, the network time protocol, as described above. Server configurations of most operating systems run the ntp daemon by default, but many desktop setups may not.

The sample broker, like the Power TAC simulation server and the tournament manager, is a Spring application. It can be run from a command-line using Maven, or it can be run from a development environment like Eclipse or STS, the Spring Tool Suite. STS is Eclipse with a number of additions specific to Spring, but the system builds and runs in the latest Eclipse without problems. Other IDEs can also be used if you prefer, but STS is what most of the server development team uses, with the result that there is help available within the Power TAC community.

Build and run the sample broker

The first thing to try after downloading the sample broker is to build and test it. You should do this first from the command-line using Maven as

 mvn clean test

This will download a number of modules, then build the sample broker and run a few JUnit tests. If this does not work, then you most likely have a problem with either your Java or Maven setup.

The next step is to load the sample broker code into your IDE, then make sure you can build and run it there. In Eclipse, you do this with Import->Existing Maven project, then choose the directory where you unpacked the sample broker code. You should now be able to build and test the agent code with your IDE.

When you are ready to run your broker, details on the command-line arguments are given in the README file contained in the source distribution. You can, of course, use those same arguments inside your IDE by setting up "run configurations". This will allow you to set breakpoints and debug your broker. If you want to use breakpoints, you may want to make the server pause and wait when the broker hits a breakpoint. This can be done by (1) adding the property server.brokerPauseAllowed = true to the server configuration file in server-distribution/config/server.properties, and (2) using the command-line argument --interactive when starting the broker.

Finally, instructions for running your agent with the simulation server are given in the section on running the server. Note that the broker can be set up to run multiple "sessions" or games in sequence without being re-started. This allows around-the-clock tournament schedules without requiring grad students to stay up all night just to run their code.

Modify the sample broker

The sample broker (javadoc) is a Spring application based on three major modules.

  1. The common module in the org.powertac.common package and sub-packages is shared with the simulation server, and contains the elements of the Power TAC domain model along with some basic shared infrastructure such as repositories, xml serialization code, and a configuration engine based on the Apache Commons configuration library. Source is available in the powertac-core repo on github.
  2. The broker core module in org.powertac.samplebroker.core contains the broker framework and the interface to the simulation server, including the broker-server communication protocol. Source is included in the powertac-core repo.
  3. A set of example behaviors implemented as Spring service classes in org.powertac.samplebroker, including the ContextManager, the PortfolioManager, and the MarketManager. These implement a basic set of agent behaviors, driven primarily by incoming message traffic from the server. They are intended to demonstrate recommended processes for handling messages, for per-timeslot activation, and for initialization and configuration. You are welcome to modify them or replace them with your own designs. Note that message handling is done with overloaded handleMessage() methods that are dispatched from the core module using Java reflection, and configuration is based on an annotation scheme.

The recommended way to build your broker is to modify or replace the source code in the org.powertac.samplebroker package, or to use this code as inspiration for your own code in your own packages. Although it is not absolutely necessary, we recommend that you build your broker as a maven project, or using some other tool that uses maven repositories, such as Gradle. This will eliminate the necessity to keep track of all the dependencies on Spring and Apache components. The org.powertac.common and org.powertac.samplebroker.core modules will be retrieved as needed by Maven. You can download sources from github for the powertac-core modules and import them into your IDE (make sure you import them as maven projects, otherwise you will not see their dependencies. Release downloads are available under the "Releases" tab, and the current development snapshot is available either by cloning or downloading a zip file near the upper-right corner of the page.

The broker agent is configured by adding name=value pairs to src/main/resources/config/broker.properties. You can also specify a separate configuration file using the -config option when you start the broker. Each property name is composed of the classname (without the org.powertac prefix) followed by the attribute name of a configurable value. Configurable values are instance variables or setter methods annotated in the source with @ConfigurableValue along with type and a description. Configuration happens when a service calls propertiesService.configureMe(this). See the configurable values and the init() method in PortfolioManager for some examples. If you want to configure classes that are not in org.powertac, you can simply use the entire qualified classname.

All interaction with the simulation server is through xml-serialized jms messages. You can write code to receive them by writing a handleMessage(msg) method to process each message type you want to see. All messages are instances of classes in packages org.powertac.common and org.powertac.common.msg in the common submodule of powertac-core. You may, of course, add additional message-handling modules simply by following the pattern in the existing MarketManagerService and PortfolioManagerService (see MarketManagerService.java for an example). Any module that implements the Initializable interface will have its initialize(BrokerContext context) method called before each simulation session, and all the handleMessage() methods will be registered to receive their respective message types. Outgoing messages are sent to the server by calling the sendMessage(msg) method on the context.

The server sends its per-timeslot messages out in batches, and the last one is guaranteed to be a TimeslotComplete message. When that message arrives, all modules that implement the Activatable interface will have their activate() methods called, in no particular order. If anyone feels it is important to control the order of activation, it would require a relatively easy modification in the core, along with some additional configuration. You may request such a change, report defects, or recommend other improvements, through the Power TAC issue tracker.

Brokers also need to interact with the Power TAC Tournament Manager, through a REST interface. This interaction is encapsulated in the samplebroker.core.BrokerTournamentService class in the broker-core submodule. The only purpose of this interaction is to acquire credentials and access details for simulation servers that run under control of the Tournament Manager. You should not need to modify this code.

If you want to add a new top-level module to the broker, it should be enough to simply annotate it with @Service ahead of the class declaration, implement the Initializable interface (provides per-session module initialization), and possibly implement the Activatable interface (provides per-timeslot activation). MarketManagerService and PortfolioManagerService are annotated this way. This is all the information Spring needs to instantiate it and wire it in through dependency injection, as long as the module is in a package Spring knows about. That bit of configuration is in src/main/resources/broker.xml. As shipped, all Service modules in org.powertac or any subpackage are included by Spring. If you want to use your own packages, you simply need to add another component-scan clause listing the package name at the top of your own package hierarchy.

The sample code synchronizes access to variables between the handleMessage() methods and the activate() method in the service modules. This is necessary because the handleMessage() methods run in threads managed by the JMS service, while the activate() methods run in a thread driven by the simulation clock. Failure to synchronize access to data that is shared by handleMessage() and activate() methods can result in weird errors.

In a tournament situation, Broker agents are commonly started once for a series of games, which means that you have to be careful about how they re-initialize between games. Modules that implement the Initializable interface will have their initialize() methods called before the agent attempts to log in for the next game. All initialization code should be in initialize() methods, and not in a constructor or elsewhere, otherwise your agent may not recover correctly between games. It is important that your initialization code clears out all per-game data. Note that the ID counter is re-initialized at this time, so if you fail to remove instances that use automatically-generated ID values, you can encounter duplicates, another possible source of weird errors.

When a broker logs in to the simulation server, it receives in the "broker-accept" message a number that's used as a prefix for the ID values in objects created and sent to the server during a game. This is important, because the server uses those object ID values as keys in various maps. A consequence is that it does not work to keep objects (like tariff specifications, for example) over between games - if your ID prefix is different in the next game, and it often is, your object IDs can collide with objects from some other broker. The server will avoid this by rejecting these messages, so you need to either re-create those objects, or update their IDs at the beginning of each game. You can get new ID values using IdGenerator.createId() after the login is complete. A good place to do that would be in ContextManagerService.handleMessage(Competition).

Interpret and compose messages

Once our broker is logged in, all the interaction between the simulation server and your broker is through asynchronous messages. Some message types are sent by the broker to the server, some sent by the server to the broker. A few, like TariffSpecification, are sent in both directions. All of them are in either the org.powertac.common or the org.powertac.common.msg packages. See the javadocs for full details, and contribute to the discussion on this page if you have ideas on how we could improve our documentation. It's always a work in progress, and our time and resources are limited.

At login

In a development environment, the broker simply logs into the server, and the server ignores the password. In a tournament environment, the tournament manager supplies the password when a broker is assigned to a particular server, and each server is supplied with a list of assigned brokers at the start of each session.

BrokerAuthentication
Passes broker username and password to the server at the start of a session.
BrokerAccept
Passed from server to broker to indicate broker has successfully logged in. Contains three fields:
prefix is used to see the ID generator to ensure that message ID values will be unique among brokers.
key is a short message prefix used to prevent spoofing and unintended broker holdovers between successive games. This will be inserted at the front of each message sent to the server.
serverTime is the server's local time when the BrokerAccept message is generated. This can be used to estimate network latency, and to ensure that the broker's local time is not out of range.

Start of game

Once a broker has successfully logged in, the server sends details on the game that can be used to adjust parameters and make initial decisions about pricing and other market activities.

Competition
Contains information about the server and the game about to start, including the list of competing brokers, the list of customer models, the PomId or version of the server, clock parameters, timeslot parameters, parameters for wholesale and balancing markets, clock parameters. See the javadoc for details.
CustomerInfo
Inside the Competition message is a list of CustomerInfo structures. Each gives information about one of the customer models, including its population, its PowerType (whether it's a producer, consumer or storage type, whether it's curtailable, and how much regulation capacity it can offer.
CustomerBootstrapData
For each customer, gives the actual consumption/production for each timeslot of the bootstrap period.
MarketBootstrapData
Gives the amount of energy traded and prices for the bootstrap period.
SimStart
Signals the start of the simulation session, communicates the server time at the start to allow the broker to synchronize its own simulation clock.

Timeslots and timekeeping

Messages include start and end markers for each timeslot, and information about pauses, which update clock parameters.

TimeslotUpdate
Sent at the start of each timeslot, gives the first and last timeslot for which the wholesale market will accept new orders during the current timeslot.
TimeslotComplete
This is always the last message from the server for the current timeslot.
PauseRequest
If the server is configured to allow broker-initiated pauses, then this is the message sent by a broker to request a pause. If it succeeds, the server will respond with a SimPause message to all brokers.
PauseRelease
If the server is currently paused by the broker sending this message, then the pause will be released and a SimResume message will be sent to brokers.
SimPause
Sent by the server to indicate that the simulation clock is paused. This can happen if a broker requests a pause, and it can happen if the server takes too much time processing a single timeslot. This is sometimes necessary to guarantee that brokers have a minimum amount of time to make their decisions in each timeslot. The server property server.simulationClockControl.minAgentWindow sets this minimum.
SimResume
Sent by the server to indicate that a pause is released, and gives updated clock parameters to re-synchronize server and broker clocks.

Tariff market

These messages are the interface between brokers and the tariff market, and between brokers and their customers. New tariffs are normally published every six hours, at which time a fee must be paid for publication. Customers can evaluate tariffs at any time, but typically do so no more often than every six hours. Most customers ignore new tariff publications most of the time as described in the game specification.

TariffSpecification
This is a description of a new tariff offering. It is sent from a broker to the server to publish a new tariff, and sent from the server to brokers when tariffs by other brokers are published. Properties include powerType to specify which types of customers may subscribe, a signupPayment to reward (positive value) or punish (negative value) a customer for subscribing, a periodicPayment that the customer must pay daily regardless of energy production or consumption, a minDuration that specifies the minimum time a customer must stay subscribed to avoid paying an earlyWithdrawPayment (negative if the customer pays the broker), and potentially the ID of an existing tariff that supersedes this tariff. Note that the supersede relationship only applies if the superseded tariff is revoked. If expiration is specified, it is the time after which new subscriptions to this tariff will not be accepted. It is specified as a long giving the number of milliseconds since 1970-01-01T00:00:00Z, which can be computed by retrieving the current time from the TimeService, calling getMillis() on the returned Instant value, and adding the desired number of hours converted to msec. A valid tariff must include at least one Rate, and zero or one RegulationRate.
Rate
A Rate specifies the cost of power potentially over a specified time interval and under specific conditions of consumption or production. A fixed rate specifies a single value, which is the cost (negative value) to the customer for consuming a unit (one kWh) of energy. If the rate is not fixed, then it must specify a minValue, a maxValue, and an expectedMean value. The tariff market keeps track of the actual realized prices for variable-rate tariffs, so customers can use this information in their evaluations. A variable-rate must also specify a noticeInterval, which is the minimum horizon over which subscribers must be notified with VariableRateUpdate messages containing HourlyCharge instances.
A Rate can also specify when it is applicable; the dailyBegin and dailyEnd fields give hours of the day (0 .. 23) when the rate comes into effect and goes out of effect, and the weeklyBegin and weeklyEnd fields specify the days when they apply (1 .. 7). So a rate with dailyBegin = 7, dailyEnd = 12, weeklyBegin = 6, and weeklyEnd = 1 applies from 7:00 until noon on Saturdays and Sundays. If these time-of-use rates are included, the set of rates must cover all times. So if we have a rate starting at 6 and ending at noon, one starting at noon and ending at 17, one starting at 17 and ending at 23, and one starting at 23 and ending at 6, we have covered all hours. If weekly start and end days are also included, they must also cover the week.
Finally, Rates can be used to specify "tiered" or "block" pricing that charge more or less for consumption/production over some threshold in a day. So if we have a rate with a fixed value = .10 and tierThreshold = 0 and another with a fixed value = .15 and tierThreshold = 10, then the first 10 kWh during a given day will be billed at .10/kWh, and the remainder will be billed at .15/kWh.
RegulationRate
Used to specify per-kWh payments (positive values) to the customer for up-regulation (which may be achieved by reducing demand) and charges (negative values) for down-regulation. These prices become bids in the balancing market in every timeslot, and can be exercised to the extent that customers report their regulation capacity. The response field is ignored in version 1.8 and earlier, and there are currently no plans to use it.
TariffStatus
Sent to a broker to report the success or failure of new tariff publication, and of tariff updates such as TariffRevoke and TariffExpire. If the status value is not Status.success, then the new tariff or update has failed, and the status value indicates the reason for failure.
TariffTransaction
Sent to brokers to report on credits or debits related to specific tariffs. The most common transaction types are PRODUCE and CONSUME. For a tariff with periodic payments, a PERIODIC transaction is issued to represent the fixed payment. These transactions are issued by each subscribed customer in each timeslot, and for population models they also include a customerCount value to indicate the number of individual customers that are contributing to the transaction. Other transaction types include PUBLISH, indicating that a tariff has been published, SIGNUP when customers subscribe to the tariff, WITHDRAW when customers withdraw from the tariff, REVOKE to indicate that a tariff has been successfully revoked, and REFUND when a customer withdraws from a tariff that included a negative signup payment.
VariableRateUpdate
If a Rate specifies fixed = false, then it is a "dynamic-price" rate, and the customer must be informed of the future prices that will be charged over the promised noticeInterval. To accomplish this, the broker must issue a VariableRateUpdate containing an HourlyCharge for each hour in the future over the notice interval. So if the notice interval is 6 hours, then 6 VariableRateUpdate messages must be sent immediately to cover the first six hours, and one additional VariableRateUpdate must be sent in each subsequent hour. Some customer models will use this information to optimize their usage.
HourlyCharge
Specifies the per-kWh charge for a VariableRateUpdate.
EconomicControlEvent
Allows a broker to request curtailment for a specific future timeslot from a subscribed INTERRUPTIBLE_CONSUMPTION or STORAGE customer. In addition to a tariff, two fields must be specified: timeslotIndex specifies the timeslot when this control is to be applied, and curtailmentRatio specifies the amount of curtailment to be applied, as a ratio of the total flexibility currently available. The broker can use this to reduce (0 < ratio <= 1) consumption, withdraw energy from storage (1 < ratio <= 2), or increase (0 > ratio >= -1) consumption at times of extreme wholesale prices, or when it expects to be imbalanced, but of course the broker will not know the actual impact of its order because it does not know precisely how much flexibility a given customer will have in the current or future timeslots. Of course, repeated use of ratios > 0 will quickly deplete up-regulation capacity, and repeated use of ratios < 0 will quickly deplete down-regulation capacity of the customers subscribed to a given tariff.
TariffExpire
Changes the expiration time of the specified tariff, after which new subscriptions will not be allowed.
TariffRevoke
Revokes an existing tariff. Assuming that tariff has already been specified as being superseded by a new tariff, then all affected customers will be switched to the superseding tariff, but they are also notified and are free to choose some other tariff instead. In fact, revoking a tariff annoys most customers, and unless the superseding tariff is a better deal than its current competitors, customers may instead choose some other tariff. There is a fee for revoking a tariff, so a TariffTransaction will be issued to notify the broker of the fee.

Wholesale market

Order
An Order is a bid in the wholesale market. A positive quantity of energy is an offer to buy energy (in MWh), and a negative quantity of money is an offer to pay up to that amount per MWh. A negative energy quantity is an offer to sell, the price being the minimum acceptable payment per MWh. If the price is left empty, then it is a "market order" and will transact at the market clearing price. See Section 5.2 of the game specification for details on how market clearing works.
OrderStatus
This message is sent to the broker if an order could not be processed, due to the specified timeslot being out of range. So if the current timeslot is n and the wholesale market is configured to trade in timeslots n + 1 through n + 24, then an order for timeslot n would result in the server sending an OrderStatus message.
ClearedTrade
Conveys information (how much was traded at what time and price, and for which timeslot) about market clearings to all brokers for each timeslot in which energy was traded.
Orderbook
Conveys information, in the form of a set of OrderbookOrder instances, about uncleared bids and asks for each timeslot in which the market clears. This allows brokers to view unsatisfied demand and supply and to estimate the shapes of the supply and demand curves.
OrderbookOrder
Represents an uncleared Order from a market clearing, giving the requested quantity and limit price.
MarketTransaction
Informs brokers about their own cleared market transactions, giving quantity traded, price, and delivery timeslot. The actual money and energy will trade hands at the target timeslot, not at the timeslot when the order(s) cleared.
MarketPosition
Sent at the conclusion of trading in each timeslot to inform the broker of the total amount of energy that will be transfered in a specific future timeslot for which one or more of the broker's Orders have cleared.

Balancing market

BalanceReport
Reports the total net imbalance for the current timeslot. Brokers who carry an imbalance with the opposite sign (and a smaller magnitude) stand to gain when the balancing market clears.
BalancingControlEvent
Reports the amount of regulation applied to customers of a given tariff that results from the balancing process. The tariff must be for a customer of type INTERRUPTIBLE or STORAGE. Regulation can be applied in two cases: (1) the tariff contains a RegulationRate specifying a price that clears the balancing market, or (2) the broker has issued a BalancingOrder against the tariff with a price that clears the market. Note that every BalancingControlEvent will be accompanied by a TariffTransaction giving the amount of energy involved and the market clearing price, which will always be no lower than the price specified in the RegulationRate or BalancingOrder.
BalancingOrder
An offer of balancing capacity at a specified ratio and price to the Balancing Market. In most cases, these are generated internally when a new TariffSpecification containing a RegulationRate is processed, in which case the ratio is ignored. For interruptible or storage tariffs that lack regulation rates, the BalancingOrder specifies the amount of regulation that can be applied using the ratio field: if 0 < value <= 1, then consumption in the current timeslot can be curtailed by this ratio; if 1 < value <= 2, then a storage type with ability to discharge (like a battery) can be asked to discharge up to its maximum discharge power; if 0 > value >= -1, then a storage type can be asked to consume additional energy up to the difference between its maximum charge rate and the amount it otherwise would have consumed. Note that the ratio value is ignored for storage types subscribed to a tariff with a regulation rate, because in this case the subscribed customers report their available capacity directly to the balancing market.
BalancingTransaction
Reports the broker's final supply-demand imbalance in the current timeslot, and the total charge or payment for this imbalance.

Distribution utility

DistributionReport
Reports the total consumption and total production in the current timeslot across all customers.
DistributionTransaction
The Distribution utility charges brokers for their use of the grid in (potentially) three different ways: (1) flat per-customer fees (also called meter fees) for each subscribed customer; (2) capacity fees for broker contributions to peak demand; and (3) fees for net energy transport. The default configuration of the server does not charge transport fees, and meter fees are the sum of smaller per-customer fees for "small" customers, and higher fees for "large" customers. The DistributionTransaction is issued in each timeslot, giving the number of small meters, the number of large meters, and the energy transport values that are being assessed, along with the total charge.
CapacityTransaction
A large fraction of the cost of a grid is a function the peak power it must be capable of transporting. For real-world distribution utilities, this cost can be comparable to total energy costs. In Power TAC, peak capacity assessments are done periodically (every two weeks in the tournament configuration). Details on how peaks are measured and assessed are given in Section 7.2 of the game specification. During each assessment, zero or more (typically up to three) peaks are identified and individual broker contributions measured. Each broker who contributed to a peak is sent a CapacityTransaction giving the timeslot in which the peak occurred, the threshold that was exceeded, the broker's contribution to the peak (which can be negative for brokers whose customers produced more energy than they consumed in that timeslot), and the charge (or credit) for the peak.

Bank

BankTransaction
Reports interest charges and payments on a broker's bank account. Issued once/day in the standard tournament configuration.
CashPosition
Reports the broker's total bank balance, issued once/timeslot after all transactions have been processed.

Weather service

WeatherReport
Reports weather conditions for the current timeslot. Fields include temperature, windSpeed</code, windDirection, and cloudCover.
WeatherForecast
Contains a list of WeatherForecastPrediction instances, one for each of the following 24 hours.
WeatherForecastPrediction
Reports predicted weather conditions for a specified future timeslot. Fields are the same as WeatherReport.

End of game

SimEnd
Sent at the end of a simulation session. This is guaranteed to be the last message sent by the server in a session.

Power TAC simulation server

To test your broker, you will need it to interact with the server. The Power TAC simulation server (or just the "server") is a Spring application, designed to be installed and run using Maven. Note that the server release packages are simply maven configuration files with some documentation; all the actual code will be downloaded by maven the first time you try to run the server.

Information about modifying and building the server is available on the developer's wiki at github.

Running the server

The server runs the Power TAC simulation in two modes; detailed instructions are in the README file included with the server distribution package once you have downloaded and unpacked it:

  • In "bootstrap" mode, the model runs for a period of time with only the default broker, and collects data on customer power usage, market prices, and weather. This data is saved to a file at the end of the bootstrap period. The resulting data file can be used to run multiple sim sessions.
  • In "sim" mode, the model loads a bootstrap data file and restores the model to its state at the end of the bootstrap period, then allows brokers to log in and sends them the bootstrap data before starting a simulation. The simulation literally starts at the end of the bootstrap period, so for example the first timeslot is not number 0, but the number of timeslots in the bootstrap period (typically 360). Note that not all timeslot-related data is carried over from the bootstrap session - for example, you cannot see the orderbooks from timeslots that completed in the bootstrap period.

As described in the README file, you can run the server as a web-based app under the jetty webserver, or as a command-line app without a visual interface. In bootstrap mode, there is not much to see, and it typically runs too fast for the visualizer to keep up. However, you can run a bootstrap session using the visualizer to generate the bootstrap data file needed for a full simulation session.

Using the visualizer (version 1)

This version is obsolete and no longer supported.


Using the visualizer (version 2)

You can run the new version of the visualizer by issuing the Maven command

 mvn -P web2

This will start up a server on http://localhost:8080. It takes a little while for it to start up, so please be patient.

This allows you to run boot sessions, and subsequently -- given the record of a boot session -- to run simulations with brokers. You can replay earlier games (either ones that you've run locally or using the state logs from external sources). You can even provide an URL to a compressed log bundle as produced during the annual Tournament, e.g. one of the entries among the past tournaments on powertac.org, and replay that directly.

The GUI to set up and start/stop games, of all varieties, is in the Games tab of the application, and ought to be more or less self-explanatory. (You don't see all input fields at once, they become visible as you enter information).


Creating game.png


Once a game is running you can monitor its progress in the Graphs tab. Note that this page is a bit tall, scroll down to see further plots. One thing to keep in mind, when running sims, is that the game doesn't actually start until all the brokers you have added to the game have connected. Until that time, the game status will remain "WAITING".


Running game.png


Each of the plots has a small button in its top-right corner, which allows you to save the contents of the plot as a separate image file.


Export plots.png


In the Files tab you'll find a list of materials, like config files, state- and trace-logs, which you can download. In this same tab you can also upload material to be used in running or replaying games.


Files.png


All of the files, as well as the logs generated by sims/boots and the visualizer itself, are under the /files directory. If you drop in material, e.g. a configuration .properties file, into the appropriate location there, the visualizer should detect them and make them available via the GUI the next time you initialize a game.


Using the command-line interface

For many testing purposes, it makes more sense to run the server from the command line. Detailed instructions are given in the README file.

Understanding the log files

As the server runs, it dumps data to two log files, known as the trace log and the state log. See the Logfile Analysis page for details on interpreting this data.

Running your broker in a competition

Once you are ready to participate in a competition, you will need to register, and set up your broker(s) with the server-supplied credentials. These credentials are specific to a particular tournament. Detailed instructions are currently on the github wiki.