Using native Nuget packages

Nuget packages have become the de facto way for .net libraries to be consumed by other projects and interestingly, Microsoft also use this technology to supply some of their C++ libraries – specifically they use this to provide the code used to connect to Xbox Live (Microsoft.Xbox.Live.SDK.Cpp.UWP). This got us thinking as to how to best share common code between our games. This shared code is within a library called BorsukCore.

Initially, we had the following approach to a clean build:

  • Build BorsukCore (Includes our custom code, the DirectX toolkit and a few other libraries)
  • Build SpacegirlCore
  • Build SpacegirlWindowsCore
  • Build Spacegirl (Win32 build)

Note that we had an equivalent build step for the UWP build (to cover the MS store / X1) and this doesn’t show the testing projects.

This meant that not only each time that a change was made to BorsukCore, the whole build would be performed, but that was also true for each change on the main game code. This increased the turn around time for each iteration. This wasn’t too bad for SG1, but got annoying when this was SG2 came along.

To that end, we set out to change matters by creating Nuget packages for the various projects built as part of BorsukCore. We mimicked the approach of MS by having separate Nuget packages for header files and .lib files. This left us with:

  • BorsukSoftware.Games.BorsukCore – headers
  • BorsukSoftware.Games.BorsukCore.x64 – libraries for both debug and release builds

We also have similar packages for our build of GoogleTest (which we use for unit testing the C++ game code), as the implementation of these is identical to the main ones, we don’t go into the details of these.

Structure of the packages

BorsukSoftware.Games.BorsukCore

  • build
    • native
      • include (directory)
      • BorsukSoftware.Games.BorsukCore.targets

BorsukSoftware.Games.BorsukCore.x64

  • build
    • native
      • lib
        • x64
          • debug (directory of libs)
          • release (directory of libs)
    • BorsukSoftware.Games.BorsukCore.x64.targets

NB It’s really important that the names of the .targets files are the same as the containing Nuget package otherwise when one tries to add them later in Visual Studio one will get infuriating errors.

Building the packages

There are lots of articles already available on the web detailing how to build the packages. The following are probably the best starting points (we used these to guide us)

Note that article #2 is wrong when it says that VS doesn’t do anything to help. If the .targets files are named consistently with the name of the package then VS will be your friend! This point is made clear in article #3.

Uploading the packages

This entirely depends on the CI process that one uses. We have a CI job which:

  • builds the core libraries in both debug and release modes
  • Runs the unit tests
  • Creates the packages from the .nuspec files
  • Uploads them to our internal repository

Consuming the packages

Once the packages have been uploaded, then one can simply add a Nuget reference to them in VS and everything should work. VS will automatically add a link to your .targets file from the project’s file. If this doesn’t happen, then this is usually because the .targets file isn’t named consistently with the name of the Nuget package.

Summary

By moving to the nuget packages approach we were able to improve our development experience both by avoiding the need to rebuild everything on each commit (~3 minutes a time) and also making it easier to put together prototypes using the common library.

 

Installing TeamCity on Linux using Docker with Perforce support

Here at Borsuk software, we use TeamCity as our CI infrastructure. We also use perforce as our source code repository. We recently migrated our TeamCity infrastructure from Windows only (Windows server and Windows agent) to adding a Linux agent (for .net core functionality) as well to using Linux for the server (reduced Azure bill / allows for more consistent environment with the rest of our infrastructure where we use Docker).

Out of the box, TeamCity provide a docker image which can be used to set up an infrastructure very easily and the installation instructions are very comprehensive. For details, please see:

Unfortunately, as at the time of writing (27-Oct-2019), this set up doesn’t add support for using Perforce. So in order to be able to use Perforce as the source repository, there’s a little bit more work to do.

Background

When it comes to source code, TeamCity has 2 modes of operation, checking out on the server and checking out on the agent. For sheer simplicity (we have 1 server, 1 Windows agent and 1 Linux agent), the approach that we’ve taken is to do the check out on the server to minimise the number of places where we need to configure. This approach isn’t the best option for the highest performance (the server can become a bottleneck) but it does work for us.

When TeamCity interacts with Perforce, it does so through the p4 executable.

Docker images are layered (comes in handy later).

Solution

Given the information above, our approach to adding support is to create a custom Docker image for the TeamCity server which contains for the p4 executable. There are [at least] 2 ways to do this:

  1. Clone the official instructions at https://github.com/JetBrains/teamcity-docker-server and then update to add the p4 executable
  2. Create a new Dockerfile which takes the existing officially built Docker image and adds the p4 executable to it.

We chose option #2 for simplicity’s sake (as we could do this on the machine which actually runs our CI infrastructure over ssh in a rather more simple fashion) and also as it’ll be easier to consume subsequent updates from Jetbrains.

Prerequisites

  • A copy of p4
    • Can be downloaded from www.perforce.com
    • Make sure that it’s executable
  • Docker is installed

Steps

  1. Download a copy of p4
  2. Make sure that it’s executable
  3. Create the following Dockerfile in the same directory where p4 exists
  4. Build the docker file
  5. Run the Docker run command as usual – remembering to use the new name rather than jetbrains/teamcity-server
  6. When the TeamCity server is up and running, it’s necessary to tell TeamCity where to find the p4 executable, this can be done by setting the following internal property – teamcity.perforce.customP4Path. For instructions on how to do this, please see – https://www.jetbrains.com/help/teamcity/configuring-teamcity-server-startup-properties.html . Using the script below, this would mean ‘teamcity.perforce.customP4Path=/perforce/p4
FROM jetbrains/teamcity-server as BASE
RUN mkdir /perforce
COPY p4 /perforce/p4
EXPOSE 8111
CMD ["/run-services.sh"]

Note that when upgrading to use a new version of TeamCity, the following steps should be taken:

  1. Download the updated TC image – docker pull jetbrains/teamcity-server
  2. Stop your existing docker container for the server and rename optionally
  3. Optionally rename your existing image (in case you want to potentially rollback)
  4. Build the above image
  5. Start a new docker container using the new image and the same settings as initially launched – e.g. the mounts, port settings etc.

Note that, depending on the spec of the machine running the server, the initialisation of the server may taken many minutes (we’ve seen up to 20 minutes on our low spec server – Azure B2S). This appears to be disc IO bound and is probably worth us investigating at a later date. Whilst it’s doing this, the other containers that we have running on the machine tend to be fairly unresponsive. Whether this is a problem or not depends on your requirements

Conclusion

With this, we were able to migrate our TeamCity instance from Windows to Linux using Docker with hopefully it being trivially easy to take new versions of TeamCity by simply rebuilding the above Docker image everytime Jetbrains release a new version of the server (note that by using Docker, your TeamCity instance will no longer be automatically updatable from within the app).

Note that if you’re looking for instructions on how to do the migration from Windows to Linux, Jetbrains has a guide online.

If this is useful or if you have suggestions on how to make this better, do let us know in the comments.

SimpleXML Released

We’re pleased to announce the release of SimpleXML, a jQuery library to allow a website to display XML data in an interactive fashion.

It was developed as part of an internal tool to allow us to view and interpret test results more easily and we hope that it proves as useful to the wider community as it has been to us.

It’s completely free to use for any purpose and can be downloaded from our GitHub page.

To see a demo of the plugin, have a look at the product page.