Recipes

Useful Elex patterns. Contribute your own.

Filter with jq and upload to S3

This recipe uses the jq json filtering tool to create a national results json data file with a limited set of data fields and the AWS cli tools to upload the filtered json to S3.

Requirements:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
#!/bin/bash

# S3 url: MUST be set to your bucket and path.
ELEX_S3_URL='mybucket.tld/output/path.json'

# Get results and upload to S3
elex results 2012-11-06 -o json \
| jq -c '[
            .[] |
            select(.level == "state" ) |
            select(.officename == "President") |
            {
              officename: .officename,
              statepostal: .statepostal,
              first: .first,
              last: .last,
              party: .party,
              votecount: .votecount,
              votepct: .votepct,
              winner: .winner,
              level: .level
            }
         ]' \
| gzip -vc \
| aws s3 cp - s3://$ELEX_S3_URL \
    --acl public-read \
    --content-type=application/json \
    --content-encoding gzip

# Check response headers
curl -I $ELEX_S3_URL

# Get first entry of uploaded json
curl -s --compressed $ELEX_S3_URL | jq '[.[]][0]'

ELEX_S3_URL must be set to your s3 bucket and path.

Steps:

  • Get election results in json format with elex
  • Pipe results to jq for filtering
  • Pipe filtered results to gzip to compress
  • Pipe gzipped results to aws s3 cp to send to S3.

Inspect with an ORM using Flask and Peewee

This recipe uses the Flask web framework and the Peewee Python ORM to model, query and update data that elex provides.

Requirements:

  • Elex loader, an NYT project that calls elex to load data into a Postgres database with CSV and the Postgres COPY command.
  • Elex admin, an NYT project that is a simple, web-based admin for creating and editing data to override AP election results, including candidate names, race descriptions, and race calls.

Steps:

Extra steps:

  • Use the models.py that come with elex-admin to query data.