Commit Graph

27 Commits

Author SHA1 Message Date
David Roetzel
2ea9336b68
Do not pass unknown encoding names to nokogiri. (#30987) 2024-07-10 14:25:39 +00:00
David Roetzel
967505ee9b
Add size limit for all PreviewCard URLs (#30973)
Some checks are pending
Check i18n / check-i18n (push) Waiting to run
CodeQL / Analyze (javascript) (push) Waiting to run
CodeQL / Analyze (ruby) (push) Waiting to run
Check formatting / lint (push) Waiting to run
CSS Linting / lint (push) Waiting to run
JavaScript Linting / lint (push) Waiting to run
Ruby Linting / lint (push) Waiting to run
JavaScript Testing / test (push) Waiting to run
Historical data migration test / test (14-alpine) (push) Waiting to run
Historical data migration test / test (15-alpine) (push) Waiting to run
Ruby Testing / build (production) (push) Waiting to run
Ruby Testing / build (test) (push) Waiting to run
Ruby Testing / test (.ruby-version) (push) Blocked by required conditions
Ruby Testing / test (3.1) (push) Blocked by required conditions
Ruby Testing / test (3.2) (push) Blocked by required conditions
Ruby Testing / Libvips tests (.ruby-version) (push) Blocked by required conditions
Ruby Testing / Libvips tests (3.1) (push) Blocked by required conditions
Ruby Testing / Libvips tests (3.2) (push) Blocked by required conditions
Ruby Testing / End to End testing (.ruby-version) (push) Blocked by required conditions
Ruby Testing / End to End testing (3.1) (push) Blocked by required conditions
Ruby Testing / End to End testing (3.2) (push) Blocked by required conditions
Ruby Testing / Elastic Search integration testing (.ruby-version, docker.elastic.co/elasticsearch/elasticsearch:7.17.13) (push) Blocked by required conditions
Ruby Testing / Elastic Search integration testing (.ruby-version, docker.elastic.co/elasticsearch/elasticsearch:8.10.2) (push) Blocked by required conditions
Ruby Testing / Elastic Search integration testing (.ruby-version, opensearchproject/opensearch:2) (push) Blocked by required conditions
Ruby Testing / Elastic Search integration testing (3.1, docker.elastic.co/elasticsearch/elasticsearch:7.17.13) (push) Blocked by required conditions
Ruby Testing / Elastic Search integration testing (3.2, docker.elastic.co/elasticsearch/elasticsearch:7.17.13) (push) Blocked by required conditions
2024-07-09 13:11:34 +00:00
David Roetzel
97eddb5906
Fix details extraction when no title exists. (#30933) 2024-07-05 13:28:52 +00:00
David Roetzel
016c1e4e78
Improve handling of encoding problems when creating link previews (#30929) 2024-07-05 11:54:38 +00:00
David Roetzel
ff08d99d4d
Catch encoding errors when creating link previews. (#30853) 2024-06-27 14:41:03 +00:00
David Roetzel
2cab1c7b09
Improve encoding detection for link cards (#30780)
Some checks are pending
Bundler Audit / security (push) Waiting to run
Check i18n / check-i18n (push) Waiting to run
CodeQL / Analyze (javascript) (push) Waiting to run
CodeQL / Analyze (ruby) (push) Waiting to run
Check formatting / lint (push) Waiting to run
CSS Linting / lint (push) Waiting to run
Haml Linting / lint (push) Waiting to run
JavaScript Linting / lint (push) Waiting to run
Ruby Linting / lint (push) Waiting to run
JavaScript Testing / test (push) Waiting to run
Historical data migration test / pre_job (push) Waiting to run
Historical data migration test / test (14-alpine) (push) Blocked by required conditions
Historical data migration test / test (15-alpine) (push) Blocked by required conditions
Ruby Testing / build (production) (push) Waiting to run
Ruby Testing / build (test) (push) Waiting to run
Ruby Testing / test (.ruby-version) (push) Blocked by required conditions
Ruby Testing / test (3.1) (push) Blocked by required conditions
Ruby Testing / test (3.2) (push) Blocked by required conditions
Ruby Testing / Libvips tests (.ruby-version) (push) Blocked by required conditions
Ruby Testing / Libvips tests (3.1) (push) Blocked by required conditions
Ruby Testing / Libvips tests (3.2) (push) Blocked by required conditions
Ruby Testing / End to End testing (.ruby-version) (push) Blocked by required conditions
Ruby Testing / End to End testing (3.1) (push) Blocked by required conditions
Ruby Testing / End to End testing (3.2) (push) Blocked by required conditions
Ruby Testing / Elastic Search integration testing (.ruby-version, docker.elastic.co/elasticsearch/elasticsearch:7.17.13) (push) Blocked by required conditions
Ruby Testing / Elastic Search integration testing (.ruby-version, docker.elastic.co/elasticsearch/elasticsearch:8.10.2) (push) Blocked by required conditions
Ruby Testing / Elastic Search integration testing (.ruby-version, opensearchproject/opensearch:2) (push) Blocked by required conditions
Ruby Testing / Elastic Search integration testing (3.1, docker.elastic.co/elasticsearch/elasticsearch:7.17.13) (push) Blocked by required conditions
Ruby Testing / Elastic Search integration testing (3.2, docker.elastic.co/elasticsearch/elasticsearch:7.17.13) (push) Blocked by required conditions
2024-06-21 12:51:10 +00:00
Matt Jankowski
f8afa0f614
Remove unused stub json ld context (#25454) 2023-10-16 15:38:27 +02:00
Christian Schmidt
f96e4b3047
Use original URL in preview if it redirects to 4xx page (#26200) 2023-07-28 23:02:08 +02:00
Claire
ec059317fa
Fix some link previews being incorrectly generated from other prior links (#16885)
* Add tests

* Fix some link previews being incorrectly generated from different prior links

PR #12403 added a cache to avoid redundant queries when the OEmbed endpoint can
be guessed from the URL. This caching mechanism is not perfectly correct as
there is no guarantee that all pages from a given domain share the same
OEmbed provider endpoint.

This PR prevents the FetchOEmbedService from caching OEmbed endpoint that
cannot be generalized by replacing a fully-qualified URL from the endpoint's
parameters, greatly reducing the number of incorrect cached generalizations.
2021-10-21 20:39:35 +02:00
Eugen Rochko
23aeef52cc
Remove Salmon and PubSubHubbub (#11205)
* Remove Salmon and PubSubHubbub endpoints

* Add error when trying to follow OStatus accounts

* Fix new accounts not being created in ResolveAccountService
2019-07-06 23:26:16 +02:00
Renato "Lond" Cerqueira
5c5e14c816 Fix undefined method error in sidekiq (#9807)
* Fix undefined method error in sidekiq

Body can be not nil but still be empty, which causes a
`NoMethodError: undefined method `[]' for nil:NilClass` further in the
code. This checks for an empty body to avoid the issue.

* Fix codeclimate issue
2019-01-14 17:28:41 +01:00
ThibG
e709b8da0d Ignore low-confidence CharlockHolmes guesses when parsing link cards (#9510)
* Add failing test for windows-1251 link cards

* Ignore low-confidence CharlockHolmes guesses

Fixes #9466

* Fix no method error when charlock holmes cannot detect charset
2018-12-17 19:19:45 +01:00
Eugen Rochko
cb5b5cb5f7
Slightly reduce RAM usage (#7301)
* No need to re-require sidekiq plugins, they are required via Gemfile

* Add derailed_benchmarks tool, no need to require TTY gems in Gemfile

* Replace ruby-oembed with FetchOEmbedService

Reduce startup by 45382 allocated objects

* Remove preloaded JSON-LD in favour of caching HTTP responses

Reduce boot RAM by about 6 MiB

* Fix tests

* Fix test suite by stubbing out JSON-LD contexts
2018-05-02 18:58:48 +02:00
Surinna Curtis
dc786c0cf4 Support Actors/Statuses with multiple types (#7305)
* Add equals_or_includes_any? helper in JsonLdHelper

* Support arrays in JSON-LD type fields for actors/tags/objects.

* Spec for resolving accounts with extension types

* Style tweaks for codeclimate
2018-05-02 12:40:24 +02:00
MIYAGI Hikaru
eb593a5a0c Append '.test' to hostname in stub data (#7260) 2018-04-25 14:12:28 +02:00
Kazushige Tominaga
718802a05d Added FetchRemoteAccountService spec (#6456)
* Added #link_header spec

* Added #call spec

* Delete spec of private methods

* Added #call spec
2018-02-10 17:10:57 +01:00
Akihiko Odaki
e0ef7f9d79 Fix XML oEmbed support discovery (#6104) 2017-12-27 03:29:49 +01:00
ysksn
9dd5e329ab Remove empty strings (#5732) 2017-11-17 10:52:30 +09:00
unarist
1eab53ee10 Fix an error when actor json couldn't be fetched in ResolveRemoteAccountService (#4979)
* Fix an error when actor json couldn't be fetched in ResolveRemoteAccountService

* Add specs
2017-09-17 11:54:23 +02:00
nullkal
007ab330e6 Use charlock_holmes instead of nkf at FetchLinkCardService (#4080)
* Specs for language detection

* Use CharlockHolmes instead of NKF

* Correct mistakes

* Correct style

* Set hint_enc instead of falling back and strip_tags

* Improve specs

* Add dependencies
2017-07-08 22:44:31 +02:00
abcang
5e6acf9601 Fix Nokogiri::HTML at FetchLinkCardService (#4072) 2017-07-05 14:54:21 +02:00
Akihiko Odaki (@fn_aki@pawoo.net)
e674608d10 A minor change for ProviderDiscovery and spec (#3543)
* Do not default the format in ProviderDiscovery

The format should be determined when discovering, as it is in the current
implementation, and it is a flaw if it is not determined.

* Spec ProviderDiscovery
2017-06-04 14:59:40 +02:00
ThibG
8d4e7504b1 Additional specs for URI handling (#2759) 2017-05-03 20:40:14 +02:00
ThibG
bea97ea766 Add rspec to further specify FollowRemoteAccountService (#2414) 2017-05-02 23:37:26 +02:00
Eugen
17c591ffba Punycode URI normalization (#2370)
* Fix #2119 - Whenever about to send a HTTP request, normalize the URI

* Add test for IDN request in FetchLinkCardService

* Perform IDN normalization on domains before they are stored in the DB
2017-04-25 02:47:31 +02:00
Eugen Rochko
81065bc06c Adding test for ProcessFeedService 2016-10-10 16:03:38 +02:00
Eugen Rochko
b640f35621 Writing out more tests, fixed some bugs 2016-03-20 13:03:06 +01:00