Shareable Outcomes from LUX

While some of the performance characteristics and functionality of LUX relies on licensed software, there are many outcomes of the project that we expect to share openly and broadly once the platform goes into production in Spring of 2023. These include knowledge and best practices, reusable data and images, open source software and processing architectures. The design of the project and the implementation effort has taken into account the importance of both being able to swap the licensed components out for other similar systems in the future, and enable the reuse of the developed components in other environments by other institutions.

In particular, we expect to share the following results to help other organizations in their efforts to produce similar, and better, cultural heritage discovery environments.   Information may be shared via the LUX website content, the LUX discovery platform, or other means of open dissemination.

We are sharing our content:

  • Data. All of the Yale-provided metadata is licensed as CC-0 (the equivalent of public domain, in that no rights are reserved) to be re-used, re-mixed and re-distributed in any way. The records have been enriched from external sources, most of which are also CC-0, with the Getty and OCLC Vocabularies (AAT, TGN, ULAN, VIAF and FAST) requiring attribution for downstream re-use. Records that have been reconciled against these vocabularies can be easily identified. Individual records can be downloaded one by one, and bulk datasets with a standards-based synchronization mechanism will be provided. The data is provided using the ICOM CIDOC-CRM and Linked Art international standards for cultural heritage Linked Open Data.

  • Image Content. Image and Audio/Visual content referenced in LUX is already provided for re-use via IIIF, with clear licensing.

We are sharing our technology:

  • User Interface. The user interface for LUX will be open source and is built as inter-linking components for ease of re-use in other environments. It is written in Javascript using modern and well-understood frameworks with internationalization and accessibility in mind. It scales both in terms of number of records and availability of functionality, incrementally enhancing the view based on advertised services.j

  • Data Pipelines. The data harmonization, reconciliation, validation and enrichment pipeline is also available as open source for adoption by other parties. It implements several modular services to reduce the cost of adding new sources of knowledge, and uses easily available data management tools such as PostgreSQL and Redis. It produces both JSON-LD for ease of use as a document, as well as other RDF formats for ingestion into a graph database. It can be easily extended with new reconciliation services, and can remain in sync via the IIIF Change Discovery API. Participating units might also, separately, make available their implementations of mapping from source systems into the data model. 

We are sharing our understanding:

  • Technical Documentation. A lot of thinking and effort has gone into the design and implementation of LUX and its related services. Documentation about the technical aspects of LUX are available, describing everything from the general design decisions and requirements, the architectural choices made, through to the specifics of the data model, mappings from underlying source systems, and identifiable queries that can be run against this model. Evaluation reports for considered technologies will also be made available as licensing and ethical considerations permit.

  • Decisions and Processes. There are also many non-technical discussions, processes and decisions that are worth sharing. This especially includes the consideration of remediation for biased and oppressive terminology and descriptions, as well as governance and consensus building across units and collections. Descriptions of these will be made available as whitepaper-style discussion documents, along with any deliverables generated by the responsible committees.