A protocol & event-sourced database for decentralized user-siloed data written in Typescript
The Textile/Threads developers/community are active on Slack and Twitter (@textileio), join us there for news, discussions, questions, and status updates. Also, check out our blog for the latest posts and announcements.
If you think you've found a bug in Threads, please file a Github issue. Take a look at our comprehensive contributor guide for details on how to get started.
Textile's Threads Protocol and Database provides an alternative architecture for data on the web. Threads aims to help power a new generation of web technologies by combining a novel use of event sourcing, Interplanetary Linked Data (IPLD), and access control to provide a distributed, scalable, and flexible database solution for decentralized applications. Threads is backed by a great deal of research and experience, and provides protocols for securely storing and sharing content-addressable data (on IPFS), with tooling to help standardize data creation and dissemination.
The primary public API to Threads is the Threads Database. The Database aims to provide:
Plus more features on the way!
Want something specific? Take a look at our contributor guide for details on how to ask for features, or better yet, submit a PR yourself :wink:!
Underlying the Threads Database are a number of ideas and technologies, which are outlined in detail in the Threads whitepaper. These components are all housed within this mono-repo, and include a set of core modules for creating Thread identities and keys (
@textile/threads-core), as well as tooling for data encryption and encoding (
@textile/threads-encoding), networking (with support for local (
@textile/threads-network) and remote (
@textile/threads-network-client) key management), and a local-first, event-sourced storage layer (
js-threads is implemented as a series of sub-packages, all managed within the js-threads repo. Each package is separately published to npm, so that developers can pick and choose which components of
js-threads they want to consume. For example, to install the low-level network APIs along with the encoding/encryption tooling:
npm install --save @textile/threads-network @textile/threads-encoding
Similarly, one can install the local storage layer or full-fledged Database separately:
npm i --save @textile/threads-store @textile/threads-database
npm i --save interface-datastore datastore-level buffer
If you are running
js-threads in an environment that does not support WebSockets by default (such as Node), be sure to include it explicitly using something like the following at the top of your script or module:
// Add WebSocket to the global namespace on Node global.WebSocket = require('isomorphic-ws')
The tests within the underlying sub-packages of this repo provide tests that use the various components of
js-threads, and provide useful examples to explore. Additionally, there are a growing list of examples available. Complete usage examples (with authentication etc) will be added soon.
Textile also provides remote Threads APIs you can use when developing your app. See
@textile/textile (or docs.textile.io) for details on authenticating with these APIs, and how to set up your own local development peer.
This mono-repo is made up of several sub-packages, all managed by lerna. You shouldn't have to do anything special to get started, however, here are a few steps that will make it easier to develop new functionality locally.
git clone email@example.com:textileio/js-threads.git cd js-threads npm install npm run bootstrap
Then you are pretty much ready to go. You can run tests for all sub-packages from the root directory:
npm test # Or run the node- or browser-specific tests separately npm run test:node npm run test:browser
npm run build
This project also uses incremental Typescript builds, so to take advantage of that (rather than building from scratch each time) use
compile. You should notice significant speed-ups in your build process:
npm run compile
This project utilises lerna's 'independent' release mode. So each time you run
lerna publish, it will check for local updates and determine which packages have changed and need publishing. To do this locally, follow these steps:
lerna clean && lerna bootstrapto get to a clean state. If this produces new changes, then make sure you commit these (or reject them) before proceeding.
npm i -g webpack webpack-cli typescript
lernacli tool does not seem to be able to handle these at this time.
lerna publish, and sit back and wait. Ideally, this will go off without a hitch.
If it fails after creating tags and/or pushing the version updates to git, but before the actual npm publish completes, you should be able to 'recover' by running
lerna publish from-package, which will attempt to publish new releases based on the versions available on npmjs.org. If it failed before or during tag creation, you might have to start over by dropping the version updates (
git checkout -- .) and trying again. If
lerna managed to commit the changes, you can use something like
git reset --hard HEAD~1 to reset the last commit, followed by
git push --force --follow-tags (if it was pushed to remote already and you are allowed to do pushes to master), but check that the right commit was reverted first.
See the lerna docs for other things you can do to make working across multiple packages easier. Note: If you are using
lerna commands directly (as opposed to the ones wrapped in the package.json scripts), you'll want to install
lerna globally to make your life easier:
npm i -g lerna.
See https://textileio.github.io/js-threads, which includes the technical API docs for all subpackages.
PRs gratefully accepted! Please see the contributing guide for details on getting started.
Small note: If editing the README, please conform to the standard-readme specification.
MIT (c) 2019-2020 Textile