Skip to content
This repository has been archived by the owner on Apr 3, 2018. It is now read-only.

Original readme

Jump to bottom
therealadam edited this page Aug 4, 2011 · 1 revision

Events

Suppose you want to create a new social network, with the requisite activity stream, so that users can see what their friends are up to. You start simple: users can create status updates, and view a timeline of all statuses in reverse-chronological order. First you'll create some events:

# Create a connection
require 'chronologic'
chronologic = Chronologic::Client.new

# Scott and Josh update their status
chronologic.event(:status_1, :data => {:username => 'sco', :status => 'This is Scott.'})
chronologic.event(:status_2, :data => {:username => 'jw',  :status => 'This is Josh.'})

# Get all the events in the timeline
chronologic.timeline[:events]
=> [{:username => 'jw', :status=>'This is Josh.'}, {:username => 'sco', :status=>'This is Scott.'}]

Note that the keys in the :data hash are arbitrary, but all the values must be strings. There is no assumption that you'll store status updates; maybe your application calls for something more like this:

{ :actor => "sco", :verb => "joined_group", :object => "Rubyists", :created_at => "2010-07-22 18:12:30" }

Timelines

Next you'll want to view all of the events from a given user. To do that, assign events to any number of timelines:

# Create events and add them to some timelines
chronologic.event(:status_1, :timelines => [:sco], :data => {:username => 'sco', :status => 'This is Scott.'})
chronologic.event(:status_2, :timelines => [:jw],  :data => {:username => 'jw',  :status => 'This is Josh.'})

# Get a timeline by name
chronologic.timeline(:sco)[:events]
=> [{:username => 'sco', :status=>'This is Scott.'}]

Subscriptions

Next you want to follow a bunch of users, and see their activity aggregated. You could do this aggregation on-demand: fetch a bunch of users' timelines, and merge them all together. But that's a lot of wasted effort, and could be slow if you follow thousands of people. You could also do the fan-out on write: look up a user's followers, and adding each of them to the :timelines array. But what happens when the user gets a new follower? Or loses one? To help with these scenarios, use subscriptions:

# Scott follows Josh and Keegan
chronologic.subscribe(:sco_friends, :jw)
chronologic.subscribe(:sco_friends, :keeg)

# Josh and Keegan update, pushing their events to their subscribers
chronologic.event(:status_2, :subscribers => [:jw],   :data => {:username => 'jw',   :status => 'This is Josh.'})
chronologic.event(:status_3, :subscribers => [:keeg], :data => {:username => 'keeg', :status => 'This is Keegan.'})

# Get aggregated activity from Scott's friends
chronologic.timeline(:sco_friends)[:events]
=> [{:username => 'keeg', :status=>"This is Keegan."}, {:username => 'jw', :status=>"This is Josh."}]

Note that subscriptions are useful for more than just a social graph. You might also use them to provide aggregated activity for all members of a group, disparate activity around a common object, etc. Unsubscribing will cause all of the right events to be removed from the appropriate timelines.

Objects

To display a complete activity feed, you'll probably need the user's name, image URL, and some other metadata. You could store that stuff right in the event, but that creates a lot of duplicated storage, making it difficult to deal with changes. To address these issues, use objects:

# Store a metadata object for each user (any time they're created or changed)
chronologic.object(:keeg, {:username => 'keeg',  :name => 'Keegan Jones'})

# When creating an event, include references to any object it relies on
chronologic.event(:status_3, :objects => {:user => :keeg}, :timelines => [:keeg], :data => {:status => 'This is Keegan.'})

# When the event is returned in a timeline, the associated objects' data will be included
chronologic.timeline(:keeg)[:events]
=> [{:status => "This is Keegan.", :user => {:username => 'keeg', :name => 'Keegan Jones'}}]

Objects are appropriate for any data that's needed to represent a complete feed, but that's not intrinsic to the event itself. Deleting an object will cause all of the events it depends on to be deleted.

Sub-events

Some events aren't directly part of a timeline, but attached another event -- like a comment on a post. Represent that with the :events option:

# To create a child, reference the parent event key
chronologic.event(:events => [:status_3], :data => {:status => 'Hi, Keegan!'})

An event can both a sub-event, and added to a timeline -- e.g., attach a comment to a post, and send a notification to the creator of the post.

Implementation, Performance & Scalability

All of the data is stored in Cassandra, which provides high availability, high write performance, and automatic partitioning of data across multiple nodes. So even with tons of users and tons of messages, it'll still be pretty fast to get the recent events for each user.

TODO: more detail about how the storage works, benchmarks, idempotence, Pull on Demand vs. Push on Change model

Installation & Configuration

Install chronologic:

sudo gem install chronologic

Edit your Cassandra config (conf/storage-conf.xml) to define the keyspace:

<Keyspace Name="Chronologic">
  <ColumnFamily Name="Object" CompareWith="UTF8Type" />
  <ColumnFamily Name="Subscription" CompareWith="UTF8Type" />
  <ColumnFamily Name="Event" ColumnType="Super" CompareWith="UTF8Type" CompareSubcolumnsWith="UTF8Type" />
  <ColumnFamily Name="Timeline" CompareWith="UTF8Type" />
  
  <ReplicaPlacementStrategy>org.apache.cassandra.locator.RackUnawareStrategy</ReplicaPlacementStrategy>
  <ReplicationFactor>1</ReplicationFactor>
  <EndPointSnitch>org.apache.cassandra.locator.EndPointSnitch</EndPointSnitch>
</Keyspace>

...or conf/cassandra.yml:

  • name: Chronologic replica_placement_strategy: org.apache.cassandra.locator.RackUnawareStrategy replication_factor: 1 column_families:

    • name: Object compare_with: UTF8Type

    • name: Subscription compare_with: UTF8Type

    • name: Event compare_with: UTF8Type type: Super subevents compare: UTF8

    • name: Timeline compare_with: UTF8Type

The RandomPartitioner should be used.

Start cassandra: sudo rm -rf /var/log/cassandra sudo rm -rf /var/lib/cassandra sudo mkdir -p /var/log/cassandra sudo chown -R whoami /var/log/cassandra sudo mkdir -p /var/lib/cassandra sudo chown -R whoami /var/lib/cassandra export JAVA_HOME=/System/Library/Frameworks/JavaVM.framework/Versions/1.6/Home export PATH=$JAVA_HOME/bin:$PATH bin/cassandra -f

Start the Chronologic server: rackup -s thin -p 4567

In production, you probably want to point to your chronologic config, with your cassandra hosts: CHRONOLOGIC_CONFIG=examples/config.yml rackup -s thin -p 4567 -E production

To install the client as a plugin in a Rails app, edit your Gemfile:

gem "chronologic", :require_as => ["chronologic", "chronologic/railtie"]

Installation & Configuration

Examples

See the examples directory.

Meta

Clone this wiki locally