Caching
RWX automatically provides content-based caching. If you run the same command on the same set of files, RWX will produce a cache hit rather than executing the task again. When using with filtering, you can ensure that your RWX runs are only executing the tasks which are necessary, based on the changes.
Content-based caching example
The cache key for each task is evaluated based on its inputs, regardless of whether upstream tasks are cache misses or cache hits. This means that RWX can produce a cache hit, even if one of its dependencies is a cache miss.
For example, let's start with this run definition:
tasks:
- key: write-foo-txt
run: echo foo > foo.txt
- key: hash-foo-txt
use: write-foo-txt
run: sha256sum foo.txt
The write-foo-txt task will write foo to foo.txt.
The hash-foo-txt task will calculate the sha256 hash of foo.txt.
Now, let's change the definition of the write-foo-txt task:
tasks:
- key: write-foo-txt
run: |
echo foo > foo.txt
echo finished writing foo.txt
- key: hash-foo-txt
use: write-foo-txt
run: sha256sum foo.txt
The run command for write-foo-txt has changed, so it will be a cache miss and RWX will have to execute it.
However, the input into hash-foo-txt is the same.
Both the first and second implementations of write-foo-txt result in foo being written to foo.txt, even though the commands and log output will be different.
When RWX goes to run hash-foo-txt, it will notice that it's already run sha256sum foo.txt on a foo.txt file which contains foo, and therefore it will produce a cache hit.
Determinism
RWX does not currently do anything to detect whether commands are deterministic.
If you run a command which is non-deterministic, such as using the date command, RWX will still cache it.
You may need to be mindful of this if you want tasks to re-execute.
Incremental updates
RWX's content-based caching means that a task must be re-run if any input to it changed.
Running a task from scratch can be undesirable if the task is amenable to
incremental updates, for example if it is a dependency installation task like
npm install or bundle install.
Tool caches help with incremental updates.
Opting out of task caching
In some cases, you may want to ensure that your task runs every time and does not leverage RWX's content-based caching.
RWX lets you configure this with the cache key on a task definition. You can set this statically to false or use an
expression to configure it:
on:
github:
push:
init:
triggered-by: push
cron:
- key: ensure-tests-pass-every-hour
schedule: '0 * * * *' # every hour
init:
triggered-by: cron
base:
os: ubuntu 24.04
tag: 1.1
tasks:
- key: run-my-tests
run: ./run-my-tests.sh
cache: ${{ init.triggered-by != "cron" }}
Setting TTLs
In certain situations, you might want the benefits of caching while still ensuring that a task occasionally runs to freshen the cache. For example, consider a task that fetches the latest version of a CLI tool:
tasks:
- key: install-rwx-cli
run: |
tmp="$(mktemp -d)/rwx"
curl -o "$tmp" -fsSL "https://github.com/rwx-cloud/cli/releases/download/v1/rwx-linux-x86_64"
sudo install "$tmp" /usr/local/bin
rm "$tmp"
rwx --version
By default, since this task has no dependencies and no expressions, as long as this task continues to be used it will be
cached indefinitely. If at the time it originally ran, the RWX CLI was at version v1.1.0 but it has since released
v1.2.0, you would be stuck using v1.1.0 unless you explicitly evicted the cache.
To get around this, you can configure a TTL on tasks. By doing so, you can specify how often the cache is automatically busted:
tasks:
- key: install-rwx-cli
run: |
tmp="$(mktemp -d)/rwx"
curl -o "$tmp" -fsSL "https://github.com/rwx-cloud/cli/releases/download/v1/rwx-linux-x86_64"
sudo install "$tmp" /usr/local/bin
rm "$tmp"
rwx --version
cache:
ttl: 7 days
RWX supports ttl specifications in the following formats:
1 min,1 minute,2 mins, or2 minutes1 hr,1 hour,2 hrsor2 hours1 dayor2 days