Structure of the Challenge

Every challenge is a git repository that you can either clone, debug, test and run locally, or use our Cloud Development Environment to run, debug and test the app.

Each challenge is your own private fork. We can read (or git clone) or write to it (or git push). Your objective is to edit the source code to patch the security vulnerability (or vulnerabilities) within the allocated time. The points and the number of tries are also displayed on the challenge page.

file explorer

Anytime we save or push the code, it pushes our changes to our remote repository. The server then runs a number of tests against our code. If tests pass, we will be given the score of the challenge. Otherwise, we can try again if the challenge timer has not ended.

Code and tests

Each challenge has at least the following directories:

  • src/ - contains the main application files

  • tests/ - contains the usability and security tests

By reading (and running the tests), we can learn how the app should behave.

Usability tests

The usability tests ensure your patch didn’t break the functinoalities of the application. Tests are found in the source folder.

The usability tests must always pass.

To run the usability tests you can use the make test command in the terminal. This will run the tests and give you the output.

Security tests

Security Test (hacker tests): These tests assesses if the vulnerability is fixed, while tring to exploit it. These tests may or may not be provided to you depending on the complexity of the challenge.

To run the security tests you can use the make securitytest command in the terminal. If you did not address the vulnerability the tests will fail.

To make the security tests pass, you need to address the security vulnerability.

The security tests are examples of an adversary attempt in exploiting our app. They help us to find where to look for security issue in the app. The security tests, teach us conditions that result into a security vulnerability. Our goal is to fix the app’s main code to prevent root cause of those condition.

Once we find and fix the security bug, we will see the security tests also pass.

Remember, we don’t change the tests! All these tests will be overwritten when we save or push our code. We make change to the app’s main code so it passes security tests while don’t break the usability tests.

The tests that runs on the server are example of the test suits (usability and security) that are given in our repository. In trivial challenges, all the security tests that server runs are given in our repository. In harder challenges, some security tests are not given in our repository.

There are also some other files in our repository that we usually don’t touch:

  • Dockerfile container definition to run the app locally on our workstation

  • Readme.adoc player guides to run and test the app. It is written in concise AsciiDoc format.

  • Makefile instruction for make command to facilitate building, running and testing the containerised app locally on our workstation. It is like a helper script.

Finding and Fixing the vulnerability

As mentioned previously, in our repository we can find a directory called src. In addition to the test suites, the src folder also contains the main application files.

An app structure depends on the language, i.e Python has a main.py while Java might have its main code tucked away in Application.Java You are encouraged to explore the src directory and understand how the application works.

You can refer to the usability tests as a specification that defines the app’s functionality. Understanding the operation of the application helps in identifying the vulnerability.

Most challenges (with exceptions like AWS and Docker) will have insecure code. There may be one or more instances of insecure programming that results in the vulnerability. By identiying the key problem areas, you must harden the code through implementing proper secure programming principles and best practices.

To edit any of the files in the File Explorer, simply select any of the files and click on the Edit pencil icon on the top right. This will allow you to modify the file and thus implement your solution for fixing the vulnerability.

Once you are done with your changes, you can commit (save) your changes by clicking on the commit button on the bottom left. Pressing Commit Only will push a commit that will be available for testing in the commit table on the challenge page. Pressing Test in the commits table will kick off the aforementioned usability and security tests to test your application with the changes made.

Commit & Test performs both these operations (pushing and testing) simultaneously. Commit Only is an option should you wish to avoid running tests immediately and have other changes to make.

Once the tests start, there will be a terminal that pops up along with the test results. Accordingly, if your patch hasn’t broken the app and addresses the vulnerability then the test will pass. Passing the tests means a successful completion of the challenge and awarding of the score.

High difficulty challenges often don’t have test output available