Commit Graph

225 Commits

Author SHA1 Message Date
Emelia Smith
f6fb9b75db
Streaming: Fix BIND IPv6 handling (#31624)
Some checks failed
Check i18n / check-i18n (push) Has been cancelled
CodeQL / Analyze (javascript) (push) Has been cancelled
CodeQL / Analyze (ruby) (push) Has been cancelled
Check formatting / lint (push) Has been cancelled
JavaScript Linting / lint (push) Has been cancelled
Ruby Linting / lint (push) Has been cancelled
JavaScript Testing / test (push) Has been cancelled
Historical data migration test / test (14-alpine) (push) Has been cancelled
Historical data migration test / test (15-alpine) (push) Has been cancelled
Ruby Testing / build (production) (push) Has been cancelled
Ruby Testing / build (test) (push) Has been cancelled
Ruby Testing / test (.ruby-version) (push) Has been cancelled
Ruby Testing / test (3.1) (push) Has been cancelled
Ruby Testing / test (3.2) (push) Has been cancelled
Ruby Testing / Libvips tests (.ruby-version) (push) Has been cancelled
Ruby Testing / Libvips tests (3.1) (push) Has been cancelled
Ruby Testing / Libvips tests (3.2) (push) Has been cancelled
Ruby Testing / End to End testing (.ruby-version) (push) Has been cancelled
Ruby Testing / End to End testing (3.1) (push) Has been cancelled
Ruby Testing / End to End testing (3.2) (push) Has been cancelled
Ruby Testing / Elastic Search integration testing (.ruby-version, docker.elastic.co/elasticsearch/elasticsearch:7.17.13) (push) Has been cancelled
Ruby Testing / Elastic Search integration testing (.ruby-version, docker.elastic.co/elasticsearch/elasticsearch:8.10.2) (push) Has been cancelled
Ruby Testing / Elastic Search integration testing (.ruby-version, opensearchproject/opensearch:2) (push) Has been cancelled
Ruby Testing / Elastic Search integration testing (3.1, docker.elastic.co/elasticsearch/elasticsearch:7.17.13) (push) Has been cancelled
Ruby Testing / Elastic Search integration testing (3.2, docker.elastic.co/elasticsearch/elasticsearch:7.17.13) (push) Has been cancelled
2024-08-28 10:40:37 +00:00
Emelia Smith
4118688fba
Streaming: Refactor move database and redis logic into separate files (#31567) 2024-08-27 08:40:04 +00:00
renovate[bot]
e3516fa8cf
Update Yarn to v4.4.1 (#31575)
Some checks are pending
Bundler Audit / security (push) Waiting to run
Check i18n / check-i18n (push) Waiting to run
CodeQL / Analyze (javascript) (push) Waiting to run
CodeQL / Analyze (ruby) (push) Waiting to run
Check formatting / lint (push) Waiting to run
CSS Linting / lint (push) Waiting to run
Haml Linting / lint (push) Waiting to run
JavaScript Linting / lint (push) Waiting to run
Ruby Linting / lint (push) Waiting to run
JavaScript Testing / test (push) Waiting to run
Historical data migration test / test (14-alpine) (push) Waiting to run
Historical data migration test / test (15-alpine) (push) Waiting to run
Ruby Testing / build (production) (push) Waiting to run
Ruby Testing / build (test) (push) Waiting to run
Ruby Testing / test (.ruby-version) (push) Blocked by required conditions
Ruby Testing / test (3.1) (push) Blocked by required conditions
Ruby Testing / test (3.2) (push) Blocked by required conditions
Ruby Testing / Libvips tests (.ruby-version) (push) Blocked by required conditions
Ruby Testing / Libvips tests (3.1) (push) Blocked by required conditions
Ruby Testing / Libvips tests (3.2) (push) Blocked by required conditions
Ruby Testing / End to End testing (.ruby-version) (push) Blocked by required conditions
Ruby Testing / End to End testing (3.1) (push) Blocked by required conditions
Ruby Testing / End to End testing (3.2) (push) Blocked by required conditions
Ruby Testing / Elastic Search integration testing (.ruby-version, docker.elastic.co/elasticsearch/elasticsearch:7.17.13) (push) Blocked by required conditions
Ruby Testing / Elastic Search integration testing (.ruby-version, docker.elastic.co/elasticsearch/elasticsearch:8.10.2) (push) Blocked by required conditions
Ruby Testing / Elastic Search integration testing (.ruby-version, opensearchproject/opensearch:2) (push) Blocked by required conditions
Ruby Testing / Elastic Search integration testing (3.1, docker.elastic.co/elasticsearch/elasticsearch:7.17.13) (push) Blocked by required conditions
Ruby Testing / Elastic Search integration testing (3.2, docker.elastic.co/elasticsearch/elasticsearch:7.17.13) (push) Blocked by required conditions
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-08-26 11:01:09 +00:00
Emelia Smith
c245a2044e
Streaming: Refactor to use metrics.$name instead of destructuring (#31566) 2024-08-26 08:08:21 +00:00
renovate[bot]
176d1feadf
chore(deps): update yarn to v4.4.0 (#31274)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-08-05 07:12:47 +00:00
Emelia Smith
7490019535
Fix support for IPv6 redis connections in streaming (#31229)
Some checks are pending
Check i18n / check-i18n (push) Waiting to run
CodeQL / Analyze (javascript) (push) Waiting to run
CodeQL / Analyze (ruby) (push) Waiting to run
Check formatting / lint (push) Waiting to run
CSS Linting / lint (push) Waiting to run
JavaScript Linting / lint (push) Waiting to run
Ruby Linting / lint (push) Waiting to run
JavaScript Testing / test (push) Waiting to run
Historical data migration test / test (14-alpine) (push) Waiting to run
Historical data migration test / test (15-alpine) (push) Waiting to run
Ruby Testing / build (production) (push) Waiting to run
Ruby Testing / build (test) (push) Waiting to run
Ruby Testing / test (.ruby-version) (push) Blocked by required conditions
Ruby Testing / test (3.1) (push) Blocked by required conditions
Ruby Testing / test (3.2) (push) Blocked by required conditions
Ruby Testing / Libvips tests (.ruby-version) (push) Blocked by required conditions
Ruby Testing / Libvips tests (3.1) (push) Blocked by required conditions
Ruby Testing / Libvips tests (3.2) (push) Blocked by required conditions
Ruby Testing / End to End testing (.ruby-version) (push) Blocked by required conditions
Ruby Testing / End to End testing (3.1) (push) Blocked by required conditions
Ruby Testing / End to End testing (3.2) (push) Blocked by required conditions
Ruby Testing / Elastic Search integration testing (.ruby-version, docker.elastic.co/elasticsearch/elasticsearch:7.17.13) (push) Blocked by required conditions
Ruby Testing / Elastic Search integration testing (.ruby-version, docker.elastic.co/elasticsearch/elasticsearch:8.10.2) (push) Blocked by required conditions
Ruby Testing / Elastic Search integration testing (.ruby-version, opensearchproject/opensearch:2) (push) Blocked by required conditions
Ruby Testing / Elastic Search integration testing (3.1, docker.elastic.co/elasticsearch/elasticsearch:7.17.13) (push) Blocked by required conditions
Ruby Testing / Elastic Search integration testing (3.2, docker.elastic.co/elasticsearch/elasticsearch:7.17.13) (push) Blocked by required conditions
2024-08-02 06:56:09 +00:00
renovate[bot]
1229d2907a
chore(deps): update docker/dockerfile docker tag to v1.9 (#31187)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-07-29 16:00:55 +00:00
Emelia Smith
c953dca1de
Streaming: use pgPool.query instead of manually acquiring & releasing a connection (#30964)
Some checks failed
Check i18n / check-i18n (push) Waiting to run
CodeQL / Analyze (javascript) (push) Waiting to run
CodeQL / Analyze (ruby) (push) Waiting to run
Check formatting / lint (push) Waiting to run
Ruby Linting / lint (push) Waiting to run
Historical data migration test / test (14-alpine) (push) Waiting to run
Historical data migration test / test (15-alpine) (push) Waiting to run
Ruby Testing / build (production) (push) Waiting to run
Ruby Testing / build (test) (push) Waiting to run
Ruby Testing / test (.ruby-version) (push) Blocked by required conditions
Ruby Testing / test (3.1) (push) Blocked by required conditions
Ruby Testing / test (3.2) (push) Blocked by required conditions
Ruby Testing / Libvips tests (.ruby-version) (push) Blocked by required conditions
Ruby Testing / Libvips tests (3.1) (push) Blocked by required conditions
Ruby Testing / Libvips tests (3.2) (push) Blocked by required conditions
Ruby Testing / End to End testing (.ruby-version) (push) Blocked by required conditions
Ruby Testing / End to End testing (3.1) (push) Blocked by required conditions
Ruby Testing / End to End testing (3.2) (push) Blocked by required conditions
Ruby Testing / Elastic Search integration testing (.ruby-version, docker.elastic.co/elasticsearch/elasticsearch:7.17.13) (push) Blocked by required conditions
Ruby Testing / Elastic Search integration testing (.ruby-version, docker.elastic.co/elasticsearch/elasticsearch:8.10.2) (push) Blocked by required conditions
Ruby Testing / Elastic Search integration testing (.ruby-version, opensearchproject/opensearch:2) (push) Blocked by required conditions
Ruby Testing / Elastic Search integration testing (3.1, docker.elastic.co/elasticsearch/elasticsearch:7.17.13) (push) Blocked by required conditions
Ruby Testing / Elastic Search integration testing (3.2, docker.elastic.co/elasticsearch/elasticsearch:7.17.13) (push) Blocked by required conditions
Bundler Audit / security (push) Has been cancelled
CSS Linting / lint (push) Has been cancelled
Haml Linting / lint (push) Has been cancelled
JavaScript Linting / lint (push) Has been cancelled
JavaScript Testing / test (push) Has been cancelled
2024-07-12 12:23:09 +00:00
Emelia Smith
d436696f46
Streaming: Fix incorrect type definitions (#30977)
Some checks are pending
Check i18n / check-i18n (push) Waiting to run
CodeQL / Analyze (javascript) (push) Waiting to run
CodeQL / Analyze (ruby) (push) Waiting to run
Check formatting / lint (push) Waiting to run
CSS Linting / lint (push) Waiting to run
JavaScript Linting / lint (push) Waiting to run
Ruby Linting / lint (push) Waiting to run
JavaScript Testing / test (push) Waiting to run
Historical data migration test / test (14-alpine) (push) Waiting to run
Historical data migration test / test (15-alpine) (push) Waiting to run
Ruby Testing / build (production) (push) Waiting to run
Ruby Testing / build (test) (push) Waiting to run
Ruby Testing / test (.ruby-version) (push) Blocked by required conditions
Ruby Testing / test (3.1) (push) Blocked by required conditions
Ruby Testing / test (3.2) (push) Blocked by required conditions
Ruby Testing / Libvips tests (.ruby-version) (push) Blocked by required conditions
Ruby Testing / Libvips tests (3.1) (push) Blocked by required conditions
Ruby Testing / Libvips tests (3.2) (push) Blocked by required conditions
Ruby Testing / End to End testing (.ruby-version) (push) Blocked by required conditions
Ruby Testing / End to End testing (3.1) (push) Blocked by required conditions
Ruby Testing / End to End testing (3.2) (push) Blocked by required conditions
Ruby Testing / Elastic Search integration testing (.ruby-version, docker.elastic.co/elasticsearch/elasticsearch:7.17.13) (push) Blocked by required conditions
Ruby Testing / Elastic Search integration testing (.ruby-version, docker.elastic.co/elasticsearch/elasticsearch:8.10.2) (push) Blocked by required conditions
Ruby Testing / Elastic Search integration testing (.ruby-version, opensearchproject/opensearch:2) (push) Blocked by required conditions
Ruby Testing / Elastic Search integration testing (3.1, docker.elastic.co/elasticsearch/elasticsearch:7.17.13) (push) Blocked by required conditions
Ruby Testing / Elastic Search integration testing (3.2, docker.elastic.co/elasticsearch/elasticsearch:7.17.13) (push) Blocked by required conditions
2024-07-10 15:59:18 +00:00
Essem
1af6313ced
Fix CMD syntax in streaming Dockerfile (#30795) 2024-06-24 10:36:26 +00:00
Essem
54cc204473
Use WebSocketServer instead of WebSocket.Server in streaming (#30788) 2024-06-24 10:29:00 +00:00
Michael Stanclift
4743657ba2
Fix Docker buildx warnings about casing and undefined variables (#30798)
Some checks failed
Bundler Audit / security (push) Has been cancelled
Check i18n / check-i18n (push) Has been cancelled
CodeQL / Analyze (javascript) (push) Has been cancelled
CodeQL / Analyze (ruby) (push) Has been cancelled
Check formatting / lint (push) Has been cancelled
CSS Linting / lint (push) Has been cancelled
Haml Linting / lint (push) Has been cancelled
JavaScript Linting / lint (push) Has been cancelled
Ruby Linting / lint (push) Has been cancelled
JavaScript Testing / test (push) Has been cancelled
Historical data migration test / pre_job (push) Has been cancelled
Ruby Testing / build (production) (push) Has been cancelled
Ruby Testing / build (test) (push) Has been cancelled
Historical data migration test / test (14-alpine) (push) Has been cancelled
Historical data migration test / test (15-alpine) (push) Has been cancelled
Ruby Testing / test (.ruby-version) (push) Has been cancelled
Ruby Testing / test (3.1) (push) Has been cancelled
Ruby Testing / test (3.2) (push) Has been cancelled
Ruby Testing / Libvips tests (.ruby-version) (push) Has been cancelled
Ruby Testing / Libvips tests (3.1) (push) Has been cancelled
Ruby Testing / Libvips tests (3.2) (push) Has been cancelled
Ruby Testing / End to End testing (.ruby-version) (push) Has been cancelled
Ruby Testing / End to End testing (3.1) (push) Has been cancelled
Ruby Testing / End to End testing (3.2) (push) Has been cancelled
Ruby Testing / Elastic Search integration testing (.ruby-version, docker.elastic.co/elasticsearch/elasticsearch:7.17.13) (push) Has been cancelled
Ruby Testing / Elastic Search integration testing (.ruby-version, docker.elastic.co/elasticsearch/elasticsearch:8.10.2) (push) Has been cancelled
Ruby Testing / Elastic Search integration testing (.ruby-version, opensearchproject/opensearch:2) (push) Has been cancelled
Ruby Testing / Elastic Search integration testing (3.1, docker.elastic.co/elasticsearch/elasticsearch:7.17.13) (push) Has been cancelled
Ruby Testing / Elastic Search integration testing (3.2, docker.elastic.co/elasticsearch/elasticsearch:7.17.13) (push) Has been cancelled
2024-06-21 20:26:39 +00:00
renovate[bot]
4e6db5e284
chore(deps): update yarn to v4.3.1 (#30790)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-06-21 11:38:26 +00:00
renovate[bot]
5b938e6e20
chore(deps): update dependency @types/uuid to v10 (#30787)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-06-21 11:37:15 +00:00
renovate[bot]
689ba28b07
chore(deps): update docker/dockerfile docker tag to v1.8 (#30736)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-06-18 12:24:14 +00:00
renovate[bot]
54ab70dfcf
chore(deps): update yarn to v4.3.0 (#30644)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-06-12 12:42:04 +00:00
Yamagishi Kazutoshi
f0f144e96d
Add customManagers:dockerfileVersions to renovate.json5 (#30607) 2024-06-10 14:47:59 +00:00
renovate[bot]
a6e0e167a7
fix(deps): update dependency uuid to v10 (#30624)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-06-10 14:29:11 +00:00
renovate[bot]
0a4a93038f
Update Yarn to v4.2.2 (#30220)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-05-10 09:22:47 +00:00
renovate[bot]
6167894547
Update dependency pino-http to v10 (#30191)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-05-06 17:14:46 +00:00
renovate[bot]
4b2054ee57
Update Yarn to v4.2.1 (#30153)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-05-02 21:22:26 +00:00
renovate[bot]
ac82f34f7d
Update dependency pino to v9 (#30057)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-04-26 14:49:54 +00:00
renovate[bot]
c70c39cad0
Update docker/dockerfile Docker tag to v1.7 (#27993)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-04-02 13:47:15 +00:00
renovate[bot]
1df00d4e76
Update dependency pino-pretty to v11 (#29662)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-03-21 10:46:14 +00:00
renovate[bot]
0f7f257139
Update Yarn to v4.1.1 (#29508)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-03-05 09:39:12 +00:00
renovate[bot]
dc4427dc9b
Update devDependencies (non-major) (#29089)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Renaud Chaput <renchap@gmail.com>
2024-02-27 16:07:43 +00:00
Renaud Chaput
899eac1a92
Use modern ES syntax rather than .call (#29368) 2024-02-27 15:42:05 +00:00
Renaud Chaput
036f5a05e3
Convert the streaming server to ESM (#29389)
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2024-02-27 14:59:20 +00:00
Emelia Smith
491dd97642
Streaming: refactor to custom Error classes (#28632)
Co-authored-by: Renaud Chaput <renchap@gmail.com>
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2024-02-22 13:20:20 +00:00
Matt Jankowski
581f14e56f
Update yarn to version 4.1.0 (#29339) 2024-02-21 21:18:01 +00:00
Emelia Smith
881e8c113c
Refactor: fix streaming postgresql and redis typing issues (#28747) 2024-01-25 16:46:02 +00:00
renovate[bot]
61a0ec69fc
chore(deps): update devdependencies (non-major) (#28840)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Renaud Chaput <renchap@gmail.com>
2024-01-23 11:44:50 +00:00
renovate[bot]
62001b5a31
Update dependency jsdom to v24 (#28836)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-01-22 11:04:28 +00:00
Emelia Smith
3fbf01918f
Streaming: Move more methods to the utils from the main file (#28825) 2024-01-22 10:02:26 +00:00
Emelia Smith
1335083bed
Streaming: replace npmlog with pino & pino-http (#27828) 2024-01-18 18:40:25 +00:00
Emelia Smith
58830be943
Streaming: Rework websocket server initialisation & authentication code (#28631) 2024-01-15 10:36:30 +00:00
Renaud Chaput
923fc83190
Improve lint-staged config (#28704) 2024-01-12 09:09:57 +00:00
Emelia Smith
9826b7780a
Streaming: use standard cors package instead of custom implementation (#28523) 2024-01-04 09:18:03 +00:00
Christian Winther
01df9f1abb
install "wget" in streaming Dockerfile (#28497) 2023-12-29 10:15:10 +00:00
Michael Stanclift
c40cfc5d09
Fix apt cache not being properly utilized in Dockerfile (#28115) 2023-11-28 22:23:44 +00:00
Emelia Smith
a4de0e364b
Refactor streaming to simplify for logging change (#28056) 2023-11-28 14:24:41 +00:00
Emelia Smith
4949b6da58
Fix streaming eslint configuration (#28055) 2023-11-28 14:09:21 +00:00
Michael Stanclift
a80530d1df
Dockerfile rewrite based on Ruby image with performance optimizations and size reduction, dedicated Streaming image (#26850)
Co-authored-by: “Michael <“mx@vmstan.com>
Co-authored-by: Emelia Smith <ThisIsMissEm@users.noreply.github.com>
2023-11-28 10:04:40 +00:00
renovate[bot]
73ab080608
Update dependency jsdom to v23 (#28075)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2023-11-27 10:06:40 +00:00
renovate[bot]
1ee3314f58
Update dependency @types/npmlog to v7 (#28048)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2023-11-23 10:28:52 +00:00
renovate[bot]
998f068499
Update Yarn to v4.0.2 (#27857)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2023-11-14 17:52:34 +00:00
Emelia Smith
15b2d7eec5
Split streaming server from web server (#24702) 2023-11-14 17:43:20 +00:00
Renaud Chaput
774e1189d2
Change eslint config to autofix missing comma and indentation in JS files (#26711) 2023-10-09 13:38:29 +02:00
Emelia Smith
f68d540271
Fix websocket connections being incorrectly decremented twice on errors (#27238) 2023-10-02 13:21:43 +02:00
Emelia Smith
fbfceb9c77
Add additional metrics for streaming (#26945) 2023-09-19 12:25:30 +02:00
Gabriel Simmer
be991f1d18
Move to ioredis for streaming (#26581)
Co-authored-by: Emelia Smith <ThisIsMissEm@users.noreply.github.com>
2023-09-01 17:44:28 +02:00
Emelia Smith
6375e390af
Fix: support both DATABASE_URL and DB_PASS (#26295) 2023-08-18 15:05:35 +02:00
Emelia Smith
65096c1ccd
Change streaming /metrics to include additional metrics (#26299) 2023-08-04 16:11:30 +02:00
Renaud Chaput
3554c52795
Fix missing return values in streaming (#26233) 2023-07-28 19:11:58 +02:00
Claire
446821a0dc
Swap debug statements in streaming server (#26231) 2023-07-28 17:59:31 +02:00
Emelia Smith
81cdc0f972
Fix: Streaming server memory leak in HTTP EventSource cleanup (#26228) 2023-07-28 12:06:29 +02:00
Emelia Smith
ddaf200c78
Refactor streaming's filtering logic & improve documentation (#26213) 2023-07-27 15:38:18 +02:00
Claire
1814990a3d
Fix wrong filters sometimes applying in streaming (#26159) 2023-07-27 15:12:10 +02:00
Emelia Smith
215081240f
Add logging of websocket send errors (#25280) 2023-06-10 18:35:57 +02:00
Emelia Smith
c81f59583c
Fix logging of messages that are binary before closing their connection (#25361) 2023-06-10 18:24:37 +02:00
Emelia Smith
16dd3f08c1
Fix performance of streaming by parsing message JSON once (#25278) 2023-06-09 19:29:16 +02:00
Renaud Chaput
d27216dc46
Enforce import order with ESLint (#25096) 2023-05-23 17:15:17 +02:00
Nick Schonning
f22b4e556c
Add eslint-plugin-jsdoc (#24719) 2023-04-30 02:29:54 +02:00
Emelia Smith
301e03eb8c
Remove clustering from streaming API (#24655) 2023-04-26 11:37:51 +02:00
Claire
174352716a
Disable anonymous access to the streaming API (#23989) 2023-03-06 21:00:10 +01:00
Eugen Rochko
be1792e1d4
Fix streaming API not being usable without DATABASE_URL (#23960) 2023-03-05 01:52:12 +01:00
Ramūns Usovs
0872f3e3d7
Allow streaming to connect to postgress with self-signed certs (#21431) 2023-03-03 21:01:18 +01:00
emilweth
7c45145430
fix metrics format (#23520) 2023-02-11 03:00:44 +01:00
emilweth
c1b823368c
dot is not allowed (#23519) 2023-02-11 02:15:07 +01:00
Eugen Rochko
c5a4d8c82d
Add information about database pool to metrics endpoint in streaming API (#23469) 2023-02-09 11:24:16 +01:00
Eugen Rochko
4cb7647516
Add database pool configuration over DB_POOL to streaming API (#23470)
Add connection timeout of 15s
2023-02-09 11:20:59 +01:00
Eugen Rochko
01584f03e8
Add metrics endpoint to streaming API (#23388) 2023-02-03 21:29:23 +01:00
Nick Schonning
06b68490d1
Enable eslint:recommended ruleset (#22433)
* Enable ESLint recommended ruleset

* Disable failing ESLint recommended rules

* Remove rules shadowed by eslint:recommended
2022-12-19 00:51:37 +09:00
Claire
8c3c6dcae9
Fix filters from other users being used in the streaming service (#20719) 2022-11-15 02:09:58 +01:00
Claire
cd5e98dbdb
Fix public/local timeline posts not being properly filtered (#20567)
* Fix streaming server using wrong property name for matching filters

Late in the PR, the `filter_results` property has been renamed to `filtered`,
but the change has not been reflected in the streaming server code.

* Fix filter_action attribute being an integer instead of a string
2022-11-13 20:59:49 +01:00
Eugen Rochko
e7aa2be828
Change how hashtags are normalized (#18795)
* Change how hashtags are normalized

* Fix tests
2022-07-13 15:03:28 +02:00
Claire
02851848e9
Revamp post filtering system (#18058)
* Add model for custom filter keywords

* Use CustomFilterKeyword internally

Does not change the API

* Fix /filters/edit and /filters/new

* Add migration tests

* Remove whole_word column from custom_filters (covered by custom_filter_keywords)

* Redesign /filters

Instead of a list, present a card that displays more information and handles
multiple keywords per filter.

* Redesign /filters/new and /filters/edit to add and remove keywords

This adds a new gem dependency: cocoon, as well as a npm dependency:
cocoon-js-vanilla. Those are used to easily populate and remove form fields
from the user interface when manipulating multiple keyword filters at once.

* Add /api/v2/filters to edit filter with multiple keywords

Entities:
- `Filter`: `id`, `title`, `filter_action` (either `hide` or `warn`), `context`
  `keywords`
- `FilterKeyword`: `id`, `keyword`, `whole_word`

API endpoits:
- `GET /api/v2/filters` to list filters (including keywords)
- `POST /api/v2/filters` to create a new filter
  `keywords_attributes` can also be passed to create keywords in one request
- `GET /api/v2/filters/:id` to read a particular filter
- `PUT /api/v2/filters/:id` to update a new filter
  `keywords_attributes` can also be passed to edit, delete or add keywords in
   one request
- `DELETE /api/v2/filters/:id` to delete a particular filter
- `GET /api/v2/filters/:id/keywords` to list keywords for a filter
- `POST /api/v2/filters/:filter_id/keywords/:id` to add a new keyword to a
   filter
- `GET /api/v2/filter_keywords/:id` to read a particular keyword
- `PUT /api/v2/filter_keywords/:id` to edit a particular keyword
- `DELETE /api/v2/filter_keywords/:id` to delete a particular keyword

* Change from `irreversible` boolean to `action` enum

* Remove irrelevent `irreversible_must_be_within_context` check

* Fix /filters/new and /filters/edit with update for filter_action

* Fix Rubocop/Codeclimate complaining about task names

* Refactor FeedManager#phrase_filtered?

This moves regexp building and filter caching to the `CustomFilter` class.

This does not change the functional behavior yet, but this changes how the
cache is built, doing per-custom_filter regexps so that filters can be matched
independently, while still offering caching.

* Perform server-side filtering and output result in REST API

* Fix numerous filters_changed events being sent when editing multiple keywords at once

* Add some tests

* Use the new API in the WebUI

- use client-side logic for filters we have fetched rules for.
  This is so that filter changes can be retroactively applied without
  reloading the UI.
- use server-side logic for filters we haven't fetched rules for yet
  (e.g. network error, or initial timeline loading)

* Minor optimizations and refactoring

* Perform server-side filtering on the streaming server

* Change the wording of filter action labels

* Fix issues pointed out by linter

* Change design of “Show anyway” link in accordence to review comments

* Drop “irreversible” filtering behavior

* Move /api/v2/filter_keywords to /api/v1/filters/keywords

* Rename `filter_results` attribute to `filtered`

* Rename REST::LegacyFilterSerializer to REST::V1::FilterSerializer

* Fix systemChannelId value in streaming server

* Simplify code by removing client-side filtering code

The simplifcation comes at a cost though: filters aren't retroactively
applied anymore.
2022-06-28 09:42:13 +02:00
Yamagishi Kazutoshi
5781d1db84
Fix parsing TRUSTED_PROXY_IP (#18051) 2022-04-19 08:11:58 +01:00
Claire
f29458da1d
Fix streaming server sometimes silently dropping subscriptions (#17841) 2022-03-21 19:08:29 +01:00
Claire
6ea80ba2a2
Change streaming server error messages when failing to parse client input (#17559)
Fixes #17541

- prefix JSON parsing error message by “Error parsing message from …”
- output user id if a user is logged in, IP address otherwise
- reduce log level from error to warning when a user is logged in, and to silly
  otherwise
2022-02-16 14:37:26 +01:00
Claire
bc066d8936
Fix timeline streaming stopping for multiple sessions instead of one (#17259)
* Fix timeline streaming stopping for multiple sessions instead of one

Fixes #17256.

In updating the code for a newer version of node-redis, #17183 also broke
redis subscription management when multiple streaming clients subscribe to the
same channel.

This commit restores the redis subscription management code.

* Let node-redis actually handle the subscriptions
2022-01-07 19:50:12 +01:00
Lerk
4d1eaf3e6e
Finish update of node-redis (#17183)
* fix streaming redis client

* use console.error instead of console.log

* follow node-redis migration guide

https://github.com/redis/node-redis/blob/master/docs/v3-to-v4.md

* fix config options for node-redis

* keep indentation

* Update streaming/index.js

Co-authored-by: Yamagishi Kazutoshi <ykzts@desire.sh>

Co-authored-by: Yamagishi Kazutoshi <ykzts@desire.sh>
2021-12-25 22:55:06 +01:00
Sasha Sorokin
6c88ebfd4b
fix(streaming): req.scopes can be nullable (#16823)
When checking for required OAuth scopes, an unexpected error could
happen due to missing (null-y) req.scopes. This commit fixes that by
checking if req.scopes are present before checking if any required
scopes are present, otherwise it skips that straight to rejection.
2021-10-13 05:02:55 +02:00
Eugen Rochko
a0d4129893
Refactor notifications to go through a separate stream in streaming API (#16765)
Eliminate need to have custom notifications filtering logic in the
streaming API code by publishing notifications into a separate stream
and then simply using the multi-stream capability to subscribe to that
stream when necessary
2021-09-26 13:23:28 +02:00
Eugen Rochko
c5c46dd6ee
Fix "cb is not a function" error in streaming API server (#16134)
Third argument of `ping` is the callback

Regression from #15932
2021-05-02 14:30:26 +02:00
Eugen Rochko
aafe65a142
Change log level of worker start/end to warn in streaming API (#16110) 2021-05-01 23:19:18 +02:00
Claire
49814d5799
Switch from deprecated ClusterWS/cws to ws package (#15932)
* Switch from deprecated ClusterWS/cws to ws package

Fixes #15184

Co-authored-by: Edho Arief <me@nanaya.pro>

* Make bufferutil and utf-8-validate optional dependencies

Co-authored-by: Edho Arief <me@nanaya.pro>
2021-03-24 09:37:41 +01:00
abcang
a2da02626e
Fixed ESLint error (#15214)
* eslint --fix

* fix consistent-return

* fix promise/catch-or-return

* ignore import rule
2020-11-23 17:35:14 +01:00
Eugen Rochko
aa10200e58
Fix streaming API allowing connections to persist after access token invalidation (#15111)
Fix #14816
2020-11-12 23:05:24 +01:00
fuyu
e39d97f700
Fix crash streaming process when receive invalid json (#14859) 2020-09-22 15:30:41 +02:00
Eugen Rochko
01647b8acb
Fix destructuring error when unsubscribing without subscribing (#14566) 2020-08-12 15:36:07 +02:00
Eugen Rochko
ef057584fd
Add support for managing multiple stream subscriptions in a single connection (#14524) 2020-08-11 18:24:59 +02:00
ThibG
ddcdddd6fc
Change streaming server to treat blank redis password as password-less auth (#14135)
Fixes #14131

Our `mastodon:setup` task defaults to a blank password rather than the
absence of password, but some versions of Redis reject blank password
authentication when authentication is possible without a password.

The Ruby code only uses the Redis password when it's not blank, so
do the same for the node.js part.
2020-06-24 22:25:23 +02:00
Eugen Rochko
5d8398c8b8
Add E2EE API (#13820) 2020-06-02 19:24:53 +02:00
Takeshi Umeda
26b08a3c54
Add remote only to public timeline (#13504)
* Add remote only to public timeline

* Fix code style
2020-05-10 10:36:18 +02:00
Ben Lubar
0dfba0884e minor server-sent events fixes (#12945)
* Send output on the server-sent events stream immediately so the client sees that it was successfully opened even if it doesn't have any messages.

Fix transparent SSE streaming for the public:local and hashtag:local stream types.

* Tell caches to never store server-sent events.
2020-01-24 20:51:33 +01:00
Eugen Rochko
24552b5160
Add whitelist mode (#11291) 2019-07-30 11:10:46 +02:00
Eugen Rochko
e7353c47db
Change default interface of web and streaming from 0.0.0.0 to 127.0.0.1 (#11302) 2019-07-15 05:56:35 +02:00
ThibG
d63c3c0cef Improve streaming server security (#10818)
* Check OAuth token scopes in the streaming API

* Use Sec-WebSocket-Protocol instead of query string to pass WebSocket token

Inspired by https://github.com/kubevirt/kubevirt/issues/1242
2019-05-24 15:21:42 +02:00
abcang
ca6c93a2f5 Migrate from uws to cws (#10805) 2019-05-22 18:19:16 +02:00