1 # Welcome to my WebSub Hub
5 [WebSub](https://www.w3.org/TR/websub/) is a protocol for subscribing to content updates from publishers. The Hub is the central component which manages that relationship.
7 This Hub implementation was created with personal [self-hostable](https://indieweb.org/WebSub) deployment in mind. It is content-agnostic, supports multiple database backends, and can scale to multiple nodes for robustness and capacity.
11 This is currently a Minimum Viable Product release. Basic functionality is complete, but the administration experience may be challenging.
15 Customize configuration within `config/${NODE_ENV}.js`. All envs inherit settings from `default.js` if not specified. Environment is selected using the `NODE_ENV` value, defaulting to `development` if unset.
17 Database table initialization and schema version migrations are automated. Configure SQLite with a database file, or point PostgreSQL to a created database.
19 A user will need to be created in order to view the `/admin` pages; the `bin/authAddUser.js` script will do this.
21 An IndieAuth profile may be used to view any topics associated with that profile.
23 The bundled logger spews JSON to stdout.
25 ### Quickstart Example
27 One way of deploying this server is behind nginx, with the pm2 package to manage the server process, and a local postgres database. Some details on this are presented here as a rough guide to any parts of this stack which may be unfamiliar.
29 - Have NodeJS 12-ish available.
30 - Have PostgreSQL available.
31 - Clone the server repository.
32 ```git clone https://git.squeep.com/websub-hub```
33 - Install the production dependencies.
35 ```NODE_ENV=production npm i```
36 - Create a ```config/production.js``` configuration file. See ```config/default.js``` for available settings.
39 > // Minimum required configuration settings
42 > selfBaseUrl: 'https://hub.squeep.com/',
45 > connectionString: 'postgresql://websubhub:mypassword@localhost/websubhub',
49 - Prepare PostgreSQL with a user and database, using e.g. ```psql```.
51 > CREATE ROLE websubhub WITH CREATEDB LOGIN PASSWORD 'mypassword';
52 > GRANT websubhub TO postgres
53 > CREATE DATABASE websubhub OWNER=websubhub;
54 > GRANT ALL PRIVILEGES ON DATABASE websubhub TO websubhub;
56 > CREATE EXTENSION IF NOT EXISTS pg_stat_statements;
58 - Install the process manager, system-wide.
60 - Configure the process manager to keep the server logs from growing unbounded.
61 ```pm2 install pm2-logrotate```
62 ```pm2 set pm2-logrotate:rotateInterval '0 0 1 * *'``` (rotate monthly)
63 ```pm2 set pm2-logrotate:compress true```
64 ```pm2 startup``` (arrange to start process monitor on system boot)
65 - Launch the server, running one process per CPU, and persist it through reboots.
66 ```NODE_ENV=production pm2 start server.js --name websubhub -i max```
68 - Create an administration user.
69 ```NODE_ENV=production node bin/authUserAdd.js admin```
70 - Copy the static files to somewhere nginx will serve them from. This will vary greatly depending on your setup.
71 ```cp -rp static /home/websubhub/hub.squeep.com/html/static```
72 - Expose the server through nginx.
75 > listen 443 ssl http2;
76 > ssl_certificate /etc/ssl/nginx/server-chain.pem;
77 > ssl_certificate_key /etc/ssl/nginx/server.key;
78 > server_name hub.squeep.com;
79 > root /home/websubhub/hub.squeep.com/html
80 > try_files $uri $uri/ @websubhub;
82 > location @websubhub {
83 > proxy_pass http://websubhub$uri;
84 > proxy_set_header Host $host;
85 > proxy_set_header X-Forwarded-For $remote_addr;
86 > proxy_set_header X-Forwarded-Proto $scheme;
87 > proxy_http_version 1.1;
91 > proxy_pass http://websubhub$is_args$args;
92 > proxy_set_header Host $host;
93 > proxy_set_header X-Forwarded-For $remote_addr;
94 > proxy_set_header X-Forwarded-Proto $scheme;
95 > proxy_http_version 1.1;
100 - The Hub server should now be available!
104 A rudimentary tally of a topic's subscribers is available on the `/info?topic=topicUrl` endpoint. The topicUrl should be URI encoded. Formats available are SVG badge, JSON, and plain text, selectable by setting e.g. `format=svg` in the query parameters.
108 The Hub keeps track of three primary entities:
110 - Topics: data and metadata for a published content endpoint. Topics are unique by source URL.
111 - Subscriptions: the relationship between a client requesting content and the topic providing it. Subscriptions are unique by topic and client URL.
112 - Verifications: updates to subscriptions which are pending confirmation. Verifications are not unique, but only the most recent for any Subscription pairing will be acted upon.
114 Any tasks in progress (notably: fetching new topic content, distributing that content to subscribers, or confirming pending verifications) are doled out and managed by a cooperative advisory locking mechanism. The task queue is wrangled in the database within the `*_in_progress` tables.
116 A Hub node will periodically check for more tasks to perform, executing them up to a set concurrency limit.
120 This implementation is built atop an in-house API framework, for Reasons. It would not be hard to replace such with something more mainstream, but that is not currently a design goal.
124 - bin/ - utility scripts
126 - default.js - defines all configuration parameters' default values
127 - index.js - merges an environment's values over defaults
128 - *.js - environment specific values, edit these as needed
129 - server.js - launches the application server
131 - authenticator.js - interact with credentials and validation mechanisms
132 - common.js - utility functions
133 - communication.js - outgoing requests and associated logic
135 - base.js - abstract database class that any engine will implement
136 - errors.js - database Error types
137 - index.js - database factory
138 - schema-version-helper.js - schema migrations aide
140 - index.js - PostgreSQL implementation
141 - listener.js - notify/listen connection to support topic content caching
142 - sql/ - statements and schemas
144 - index.js - SQLite implementation
145 - sql/ - statements and schemas
146 - enum.js - invariants
147 - errors.js - local Error types
148 - link-helper.js - processes Link headers
149 - logger.js - a very simple logging class
150 - manager.js - process incoming requests
151 - service.js - defines incoming endpoints, linking the API server framework to the manager methods
152 - session-manager.js - process login/logout requests
153 - template/ - HTML content
154 - worker.js - maintains a pool of tasks in progress, for sending out updates, performing verifications, et cetera
155 - test/ - unit and coverage tests
156 - test-e2e/ - support for whole-service testing