write_club

Project Complete!

Well, complete enough for a pin. Making a social, dynamic website was one of my personal project goals, and earlier this year I was able to complete it to my satisfaction. And I gave myself a badge.

The Idea

I was motivated to practice more creative writing, and curious to find a way that would help maintain that motivation. Usually these spurts of excitement die. Gamification and social reinforcement usually do a good job of keeping things going, so that's where the idea started. How to socially gamify creative writing? Competition.

Hence, Write Club.

The concept is as follows: - Every day, a new creative writing prompt is generated. - You are encouraged to respond to the prompt and submit it. - Submitting means putting your response into the ring with others' for the same prompt. - After a prompt gains enough submissions (12), the votes begin. - Everyone is randomly assigned 3 other responses, reads them, and rank votes them. - Once voting is completed, a winner is crowned.

How I Got Started

Well, how to make the website? I went with django to support the front end, and python to bridge the gap between that and the postgresql back-end. I wasn't familiar with django, so it was a great way to learn some stuff.

Designing

The UX and flow of the website was my starting point. You log in, see a prompt, then write? There's more to it than that. Here's a sketch that I started with:

I eventually strayed a bit from v1 here, but the flow remained similar. The core experience starts with the current prompt and a list of previous writings. Clicking in to a prompt opens up a writing canvas with a submission button. Submitting locks everything down. I also thought it was important to display your accolades and writing streak right on the home page to encourage activity.

On the surface, it was easy enough to put all these pieces together. It's just four 'pages' and some buttons and windows in each one. Taking one step deeper and asking a couple 'what should happen when I click that button' questions, and the complexity began to take shape.

Logging in

Fortunately, django helped here. There's a whole package that you can use and I didn't have to do much for that. I just connected by first page to that login module and all following pages check whether or not the current session has successfully logged in.

Daily batch jobs

Generating the prompt

So every day, there should be a new prompt generated. Easy enough, just run a python script using the OpenAI API and put the output into the prompts table. A little psuedo-code for you:

FUNCTION addprompt():
    word_list = [big list of like 400 words hard-coded here]
    day_of_year = today’s day number (1–365)
    word = pick a word from word_list based on day_of_year
    prompt = "Write about ..." + word   (short creative scene)

    response = call OpenAI with the prompt
    return response text


CONNECT to Postgres database (writeclub)


 --- Generate daily prompt ---
today = current date
check if document_prompt already has a row for today
IF not:
    new_prompt = addprompt()
    INSERT new_prompt into document_prompt
ELSE:
    print "prompt exists for the day"

Worked like a charm - caused some problems though that I'll mention later.

Word Count

The users can click into a prompt, write their thing, and save it as plain text in the document table in the database. Then it came to recording the word count.

This one put my brain in a pretzel for a minute. It sounds simple, but as soon as went to 'insert word count into the database', I realized I didn't even know how to start. You create a new document, and then there is a 'word count' column attached to it in the table. Then what? Should it be that for every new word a user types, the value is updated? I could do that, but this system would be seriously bogged down if after every 'space', a db update operation needed to be executed. There are some databases that could make it easy, but I didn't want to overcomplicate it. Another situation: how would I know how many words I typed 'today' if all I did was update an existing document?

I realized that I actually cared specifically about 'daily' net word count, which gave me an opportunity for a shortcut. I could calculate the total word count at the end of each day across all my documents and then compare it with the previous day's word count to see the difference.

First, I have a 'word count staging' sql view that I can pull from at any time:

Stage daily word counts:
  - For each document updated or created in last ~24 hours
  - Count words using regex split on spaces/newlines
  - Output:
      * current_date
      * document id
      * content
      * author
      * word_count

This shows me the total word count at the current time across all my documents.

After my python prompt generation, I run some python code that pulls from that staging view and places it into a table that records those snapshots. Then there is a column that compares each day to the previous using a lag(). This is also how I create that writing streak table: I want to look at all the days that have a non-zero value for aggregate word count, put it on a timeline, and count up for every consecutive day. It's gotta start over at 0 when there is a gap.

First, create the timeline:

Select activity across all days of the year:
  - Cross join all year's calendar dates
  - For each user & document:
      * Mark 1 if created_at is present, else 0
      * Use LEAD to check next day’s activity
  - Produces daily activity timeline

The cross-join thing helps with the gaps. I need to have all days in the table even if they didn't create anything that day.

Then, create the streak numbers based on that:

Track streaks of activity per user:
  1. Collect all distinct days users created or modified documents
  2. Merge into a single timeline (creation + modification)
  3. For each date, check if it continues a streak:
      * If the gap from the previous day < 2, count as streak
      * Else reset streak
  4. Calculate streak numbers per user

Voting

The trickiest part. I had a button on the canvas page that allows you to submit your document when you are ready. That is marked in the database and the counter for 'submissions' on the global prompt goes up by one. Once that number gets up to a certain number, voting should start.

First, I had to check whether or not voting should start. I made another view for it:

Create view of prompts ready for voting:
  - Join prompts with submitted documents
  - Only include:
      * documents that have been submitted (submitted_date is not null)
      * prompts with more than 3 submissions
      * prompts where voting hasn’t started yet

Then, for all of those, I needed to start the voting and simultaneously check for any rounds of voting that needed to be concluded:

-- Initialize votes ---
get all submissions from vote_stg
group by prompt_id

FOR each prompt with submissions:
    shuffle the submissions
    FOR each submission:
        assign 3 documents for the user to judge
        INSERT vote records into document_votes

mark prompts as "vote_started"


--- Check for completed votes ---
get prompts where votes_remaining = 0

FOR each completed prompt:
    calculate scores:
        rank 1 = 6 points
        rank 2 = 3 points
        rank 3 = 1 point
    find top 3 documents
    update prompt with first, second, third place
    update documents with their rank
    mark prompt as "vote_completed"

How to assign votees to voters

So you have 12 writers and 12 documents and you want everyone to read 3 and rank vote them 1-3. That is complicated. I thought about it like this:

There is an array of writers with their own document. Each of those writers will have their own new empty 3-item array with which I can fill in the three documents they are going to read and rank. That ring I drew is how I imagined it. Take writer A's document, and assign it to the next three writers. If I do that for every writer, everyone should be assigned 3 documents that are not their own.

From there, it was just some math to assign values to votes, distribute awards, and record them on documents and display them for users.

There were of course a lot of little bugs and troubles that I ran into, but they're not worth writing about here.

Things I would change

Daily batch

There's no reason for me to generate the prompt every day. There were days when the prompt didn't run and the whole website broke because there was no prompt for the current day. I should have just generated like 300 prompts up front and written them all into the table right away.

User security

As soon as I opened up the website to the public, bots came swarming like the zombie hoard. I was seeing like dozens of new users signing up immediately - my first thought was bots, and I was right. I could have easily implemented a honeypot to avoid that but by the time I noticed, I didn't care and just shut the website down. A honeypot is just a hidden text-entry box that only a bot would fill out (since a normal human can't see it). If that box is filled in, then deny the 'user' from signing up.

Docker

I did this all in docker containers for learning purposes, but I think it was just kinda unwieldy.

Things I learned

Docker, obviously. Django. SSL.

The official Django tutorial was great - I'd recommend doing that.