Bibdata

Formerly known as MARC Liberation (since it liberates MARC data from Voyager).

Find Internal Documentation on our confluence wiki

Development and testing

Dependencies

• Postgresql (provided in development by lando)
• brew install lastpass-cli
• brew install shared-mime-info (for mimemagic gem)

Note: You need to have PostgreSQL installed in your machine and available in your path for the pg gem to compile native extensions (e.g. export PATH=$PATH:/Library/PostgreSQL/10/bin/).

Setup server

1. Install Lando from lando releases GitHub (at least 3.0.0-rrc.2)
2. Install Sidekiq Pro credentials:

lpass login emailhere
bin/setup_keys

1. Install bundler version in Gemfile.lock

gem install bundler -v '2.2.27'

1. Install bundle

bundle install

1. To start: bundle exec rake servers:start
2. For testing:
   • bundle exec rspec
3. For development:
   • bundle exec rails server
   • Access bibdata at http://localhost:3000/
   • If you will be working with background jobs in development, include your netid so you are recognized as an admin BIBDATA_ADMIN_NETIDS=yournetid bundle exec rails server
4. If you are working with background jobs in development, start sidekiq in a new tab or window
   • bundle exec sidekiq
   • To access the sidekiq dashboard, first sign into the application, then go to http://localhost:3000/sidekiq
5. To stop: bundle exec rake servers:stop or lando stop

Configure Alma keys for Development

1. lpass login emailhere
2. bundle exec rake alma:setup_keys

This will add a .env with credentials to Rails.root

ARK Caching

In order to resolve bibliographic identifiers (bib. IDs) to resources with ARKs and IIIF manifests for resources managed within digital repositories, caches are seeded and used in order to resolve the relationships between these resources.

Seeding the Cache

One may seed the cache using the following Rake Task:

rake liberate:arks:seed_cache

In development, when running commands that utilize the cache, such as commands indexing via traject, set the FIGGY_ARK_CACHE_PATH to point to spec/fixtures/marc_to_solr/figgy_ark_cache in the local environment.

export FIGGY_ARK_CACHE_PATH=spec/fixtures/marc_to_solr/figgy_ark_cache

Clearing the Cache

One may clear the cache using the following Rake Task:

rake liberate:arks:clear_cache

Tests

Ensure redis is running

To run the tests in the marc_to_solr directory set RAILS_ENV: $ RAILS_ENV=test bundle exec rspec spec/marc_to_solr

To run all the tests use the rake task, which sets some environment variables for you: $ rake spec

Semgrep

This repository uses semgrep to:

• Guard against common gotchas within bibdata
• Perform static security analysis

To run semgrep locally:

brew install semgrep
semgrep --config .semgrep.yml . # run custom bibdata rules
semgrep --config auto . # run rules from the semgrep community
semgrep --config auto --config .semgrep.yml . # run both sets of rules

Deploy

Deployment is through capistrano. To deploy a branch other than "main", prepend an environment variable to your deploy command, e.g.: BRANCH=my_feature bundle exec cap staging deploy

Locations Configuration

See: Location Changes documentation

API Endpoints

API Endpoint documentation

Alma Webhooks

see [[webhook_monitor/README.md]]

License

See LICENSE.