1 Methodology

The number of partitions, revisions, and renders (overwrites) can be specified. Iteration happens first on the revision, then on the render, then the partition. In other words, for each render of each revision, a value is written for each partition (and the iterations on the render are overwrites of the same revision).

Values are 50K in size.

See: Github source

2 Alt Write

Straightforward key-value model using a constant for the write-time (1). Values are binary, prefixed with a 4-byte integer, and a 16 byte type-1 UUID.

2.1 Sequence

2.1.1 Truncate

c-cqlsh a -e 'truncate krv.alt_data'

2.1.2 Initial load

Establish a baseline of 1e6 records
Time allowed for cluster to quiesce upon completion
Grafana
Grafana (snapshot)

alt-write \
    --host restbase-dev1001-a.eqiad.wmnet \
    --ssl \
    --cqlshrc /etc/cassandra-a/cqlshrc \
    -np 1000000 \
    -nr 1 \
    --num-renders 1 \
    --concurrency 15

2.1.3 Mixed read-overwrite

Reads
Overwrites of existing entries (revisions and renders)
- 10e6 (2 revisions * 5 renders * 1e6 partitions)
Reads continue after overwrites complete (until compaction quiesced)
Grafana
Grafana (snapshot)

alt-write \
    --host restbase-dev1001-a.eqiad.wmnet \
    --ssl \
    --cqlshrc /etc/cassandra-a/cqlshrc \
    -np 1000000 \
    -nr 2 \
    -ro 1
    --num-renders 5 \
    --concurrency 10

alt-read \
    --host restbase-dev1001-a.eqiad.wmnet \
    --ssl \
    --cqlshrc /etc/cassandra-a/cqlshrc \
    -np 1000000 \
    --runs 1000000 \
    --concurrency 20

2.1.4 Mixed offset reads, offset overwrites

Concurrent writers (overwrites), working on different parts of the keyspace
Concurrent readers, reading different parts of the keyspace
5e6 (overwrite)writes (2 processes * (5 revision * 1 renders * 500000 partitions))
Reads continue after overwrites complete (until compaction quiesced)
Grafana
Grafana (snapshot)

alt-write \
    --host restbase-dev1001-a.eqiad.wmnet \
    --ssl \
    --cqlshrc /etc/cassandra-a/cqlshrc \
    -np 500000 \
    -nr 5 \
    -ro 2
    --num-renders 1 \
    --concurrency 3

alt-write \
    --host restbase-dev1001-a.eqiad.wmnet \
    --ssl \
    --cqlshrc /etc/cassandra-a/cqlshrc \
    -np 500000 \
    -po 500000 \
    -nr 5 \
    -ro 2
    --num-renders 1 \
    --concurrency 3

alt-read \
    --host restbase-dev1001-a.eqiad.wmnet \
    --ssl \
    --cqlshrc /etc/cassandra-a/cqlshrc \
    -np 500000 \
    --runs 1000000 \
    --concurrency 15

alt-read \
    --host restbase-dev1001-a.eqiad.wmnet \
    --ssl \
    --cqlshrc /etc/cassandra-a/cqlshrc \
    -np 500000 \
    -po 500000 \
    --runs 1000000 \
    --concurrency 15

3 Key-revision-value

Modeled as a wide row using the revision as a cluster key. A timeuuid and blob attribute are overwritten as needed.

3.1 Sequence

3.1.1 Truncate

c-cqlsh a -e 'truncate krv.alt_data'
c-cqlsh a -e 'truncate krv.data'
cdsh -c restbase-dev -- c-foreach-nt clearsnapshot

3.1.2 Initial load

Establish a baseline of 1e6 records
Time allowed for cluster to quiesce upon completion
Grafana
Grafana (snapshot)

write \
    --host restbase-dev1001-a.eqiad.wmnet \
    --ssl \
    --cqlshrc /etc/cassandra-a/cqlshrc \
    -np 1000000 \
    -nr 1 \
    --num-renders 1 \
    --concurrency 15

3.1.3 Mixed read-overwrite

Reads
Overwrites of existing entries (revisions and renders)
- 10e6 (2 revisions * 5 renders * 1e6 partitions)
Reads continue after overwrites complete (until compaction quiesced)
Grafana
Grafana (snapshot)

write \
    --host restbase-dev1001-a.eqiad.wmnet \
    --ssl \
    --cqlshrc /etc/cassandra-a/cqlshrc \
    -np 1000000 \
    -nr 2 \
    -ro 1
    --num-renders 5 \
    --concurrency 10

alt-read \
    --host restbase-dev1001-a.eqiad.wmnet \
    --ssl \
    --cqlshrc /etc/cassandra-a/cqlshrc \
    -np 1000000 \
    --runs 1000000 \
    --concurrency 20

3.1.4 Mixed offset reads, offset overwrites

Concurrent writers (overwrites), working on different parts of the keyspace
Concurrent readers, reading different parts of the keyspace
5e6 (overwrite)writes (2 processes * (5 revision * 1 renders * 500000 partitions))
Reads continue after overwrites complete (until compaction quiesced)
Grafana (temporary link)

write \
    --host restbase-dev1001-a.eqiad.wmnet \
    --ssl \
    --cqlshrc /etc/cassandra-a/cqlshrc \
    -np 500000 \
    -nr 5 \
    -ro 2
    --num-renders 1 \
    --concurrency 3

write \
    --host restbase-dev1001-a.eqiad.wmnet \
    --ssl \
    --cqlshrc /etc/cassandra-a/cqlshrc \
    -np 500000 \
    -po 500000 \
    -nr 5 \
    -ro 2
    --num-renders 1 \
    --concurrency 3

read \
    --host restbase-dev1001-a.eqiad.wmnet \
    --ssl \
    --cqlshrc /etc/cassandra-a/cqlshrc \
    -np 500000 \
    --runs 1000000 \
    --concurrency 15

read \
    --host restbase-dev1001-a.eqiad.wmnet \
    --ssl \
    --cqlshrc /etc/cassandra-a/cqlshrc \
    -np 500000 \
    -po 500000 \
    --runs 1000000 \
    --concurrency 15

Table of Contents