Ivan Tashkinov [Tue, 21 Jul 2020 17:03:14 +0000 (20:03 +0300)]
[#2497] Resolved merge conflicts.
Ivan Tashkinov [Sun, 19 Jul 2020 17:05:37 +0000 (20:05 +0300)]
Merge remote-tracking branch 'remotes/origin/develop' into 2168-media-preview-proxy
# Conflicts:
# config/config.exs
# lib/pleroma/web/media_proxy/media_proxy.ex
# mix.lock
# test/web/media_proxy/media_proxy_test.exs
feld [Sat, 18 Jul 2020 16:49:30 +0000 (16:49 +0000)]
Merge branch 'fix/followredirects-hackney' into 'develop'
Don't enable Pleroma.HTTP.Middleware.FollowRedirects unless Gun is used
Closes #1966 and #1961
See merge request pleroma/pleroma!2778
href [Sat, 18 Jul 2020 11:17:38 +0000 (13:17 +0200)]
Don't enable Pleroma.HTTP.Middleware.FollowRedirects unless Gun is used
Haelwenn [Fri, 17 Jul 2020 18:54:30 +0000 (18:54 +0000)]
Merge branch 'fix-1958' into 'develop'
OpenAPI: Add :id to follower/following endpoints, fixes #1958
Closes #1958
See merge request pleroma/pleroma!2772
Alex Gleason [Fri, 17 Jul 2020 02:39:10 +0000 (21:39 -0500)]
OpenAPI: Add :id to follower/following endpoints, fixes #1958
Haelwenn [Fri, 17 Jul 2020 16:52:59 +0000 (16:52 +0000)]
Merge branch 'feature/expose-post-formats' into 'develop'
Expose the post formats in /api/v1/instance
See merge request pleroma/pleroma!2776
Mark Felder [Fri, 17 Jul 2020 15:45:41 +0000 (10:45 -0500)]
Expose the post formats in /api/v1/instance
feld [Fri, 17 Jul 2020 14:36:38 +0000 (14:36 +0000)]
Merge branch 'feature/totp-input-improvements' into 'develop'
Improve TOTP token and recovery input fields in OAuth login
See merge request pleroma/pleroma!2769
Mark Felder [Fri, 17 Jul 2020 13:19:49 +0000 (08:19 -0500)]
TOTP length is configurable, so we can't hardcode this here.
rinpatch [Fri, 17 Jul 2020 08:05:16 +0000 (08:05 +0000)]
Merge branch 'chore/issue-template-remove-tasks' into 'develop'
Bug issue template: remove choice in "Installation type"
See merge request pleroma/pleroma!2774
rinpatch [Fri, 17 Jul 2020 08:03:47 +0000 (11:03 +0300)]
Bug issue template: remove choice in "Installation type"
It made gitlab display "1 of 2 tasks completed" when one is chosen,
which is totally not what this was used for.
Mark Felder [Thu, 16 Jul 2020 18:11:03 +0000 (13:11 -0500)]
Improve TOTP token and recovery input fields in OAuth login
feld [Wed, 15 Jul 2020 23:10:52 +0000 (23:10 +0000)]
Merge branch 'refactor/gun-pool-registry' into 'develop'
Refactor gun pooling and simplify adapter option insertion
Closes #1834, #1700, and #1680
See merge request pleroma/pleroma!2479
Mark Felder [Wed, 15 Jul 2020 18:34:27 +0000 (13:34 -0500)]
Merge branch 'develop' into refactor/gun-pool-registry
feld [Wed, 15 Jul 2020 15:22:39 +0000 (15:22 +0000)]
Merge branch 'refactor/notification_settings' into 'develop'
Refactor notification settings
See merge request pleroma/pleroma!2602
Mark Felder [Wed, 15 Jul 2020 14:38:56 +0000 (09:38 -0500)]
Update FE bundle
Mark Felder [Wed, 15 Jul 2020 14:30:23 +0000 (09:30 -0500)]
Merge branch 'develop' into refactor/notification_settings
rinpatch [Wed, 15 Jul 2020 14:00:22 +0000 (14:00 +0000)]
Merge branch 'features/mrf-reasons' into 'develop'
Add rejection reason to our MRFs
See merge request pleroma/pleroma!2759
rinpatch [Wed, 15 Jul 2020 13:52:01 +0000 (13:52 +0000)]
Merge branch 'fix/oban-pruning' into 'develop'
Bring back oban job pruning
Closes #1945
See merge request pleroma/pleroma!2768
feld [Wed, 15 Jul 2020 13:44:52 +0000 (13:44 +0000)]
Merge branch 'issue/sign-authority' into 'develop'
Include port in host for signatures
See merge request pleroma/pleroma!2766
Haelwenn (lanodan) Monnier [Wed, 15 Jul 2020 13:25:33 +0000 (15:25 +0200)]
MastoAPI: fix & test giving MRF reject reasons
rinpatch [Wed, 15 Jul 2020 13:20:17 +0000 (16:20 +0300)]
Bring back oban job pruning
Closes #1945
rinpatch [Wed, 15 Jul 2020 12:58:08 +0000 (15:58 +0300)]
ConnectionPool.Worker: do not stop with an error when there is a timeout
This produced error log messages about GenServer termination
every time the connection was not open due to a timeout.
Instead we stop with `{:shutdown, <gun_error>}` since shutting down
when the connection can't be established is normal behavior.
Maksim Pechnikov [Wed, 15 Jul 2020 12:49:46 +0000 (15:49 +0300)]
Merge branch 'develop' into issue/sign-authority
href [Mon, 13 Jul 2020 08:44:36 +0000 (10:44 +0200)]
Set a default timeout for Gun adapter timeout
href [Wed, 8 Jul 2020 17:23:32 +0000 (19:23 +0200)]
Don't test tls_options in adapter helper test.
href [Wed, 8 Jul 2020 14:02:57 +0000 (16:02 +0200)]
host is now useless
href [Wed, 8 Jul 2020 13:12:09 +0000 (15:12 +0200)]
Simplify TLS opts
- `verify_fun` is not useful now
- use `customize_check_hostname` (OTP 20+ so OK)
- `partial_chain` is useless as of OTP 21.1 (wasn't there, but hackney/..
uses it)
href [Wed, 8 Jul 2020 13:02:56 +0000 (15:02 +0200)]
Go back to upstream Tesla
href [Wed, 8 Jul 2020 12:59:10 +0000 (14:59 +0200)]
Bump gun
href [Wed, 8 Jul 2020 11:58:38 +0000 (13:58 +0200)]
parentheses...
href [Wed, 8 Jul 2020 11:22:42 +0000 (13:22 +0200)]
Fix race in enforcer/reclaimer start
href [Wed, 8 Jul 2020 11:01:02 +0000 (13:01 +0200)]
Ensure connections error get known by the caller
href [Tue, 7 Jul 2020 16:56:17 +0000 (18:56 +0200)]
ConnectionPool.Worker: Open gun conn in continue instead of init
rinpatch [Mon, 6 Jul 2020 09:13:02 +0000 (12:13 +0300)]
config.exs: set gun retries to 0
The new pooling code just removes the connection when it's down,
there is no need to reconnect a connection that is just sitting idle,
better just open a new one next time it's needed
rinpatch [Wed, 1 Jul 2020 22:53:27 +0000 (01:53 +0300)]
Connection Pool: fix LRFU implementation to not actually be LRU
The numbers of the native time unit were so small the CRF was always 1,
making it an LRU. This commit switches the time to miliseconds and changes
the time delta multiplier to the one yielding mostly highest hit rates according
to the paper
rinpatch [Tue, 30 Jun 2020 15:35:15 +0000 (18:35 +0300)]
Request limiter setup: consider {:error, :existing} a success
When the application restarts (which happens after certain config
changes), the limiters are not destroyed, so `ConcurrentLimiter.new`
will produce {:error, :existing}
rinpatch [Tue, 30 Jun 2020 12:58:53 +0000 (15:58 +0300)]
FollowRedirects: Unconditionally release the connection if there is an error
There is no need for streaming the body if there is no body
rinpatch [Tue, 23 Jun 2020 12:52:57 +0000 (15:52 +0300)]
config.exs: make gun the default again
rinpatch [Tue, 23 Jun 2020 17:36:21 +0000 (20:36 +0300)]
Gun adapter helper: fix wildcard cert issues on OTP 23
See https://bugs.erlang.org/browse/ERL-1260 for more info.
The ssl match function is basically copied from mint, except
that `:string.lowercase/1` was replaced by `:string.casefold`.
It was a TODO in mint's code, so might as well do it since we don't need
to support OTP <20.
Closes #1834
rinpatch [Tue, 23 Jun 2020 12:38:45 +0000 (15:38 +0300)]
Add documentation for new connection pool settings and remove some
`:retry_timeout` and `:retry` got removed because reconnecting on failure is
something the new pool intentionally doesn't do.
`:max_overflow` had to go in favor of `:max_waiting`, I didn't reuse the key because
the settings are very different in their behaviour.
`:checkin_timeout` got removed in favor of `:connection_acquisition_wait`,
I didn't reuse the key because the settings are somewhat different.
I didn't do any migrations/deprecation warnings/changelog entries because
these settings were never in stable.
rinpatch [Wed, 17 Jun 2020 09:54:13 +0000 (12:54 +0300)]
Use erlang monotonic time for CRF calculation
rinpatch [Tue, 16 Jun 2020 13:20:28 +0000 (16:20 +0300)]
Adapter Helper: Use built-in ip address type
rinpatch [Sat, 6 Jun 2020 14:53:39 +0000 (17:53 +0300)]
Connection Pool: add tests
rinpatch [Thu, 28 May 2020 11:06:18 +0000 (14:06 +0300)]
Remove tests for old pool
rinpatch [Sun, 17 May 2020 19:16:02 +0000 (22:16 +0300)]
HTTP: Implement max request limits
rinpatch [Sat, 16 May 2020 08:49:19 +0000 (11:49 +0300)]
Use a custom pool-aware FollowRedirects middleware
rinpatch [Fri, 8 May 2020 16:57:11 +0000 (19:57 +0300)]
Connection pool: fix connections being supervised by gun_sup
rinpatch [Fri, 8 May 2020 15:18:59 +0000 (18:18 +0300)]
Connection pool: Add client death tracking
While running this in production I noticed a number of ghost
processes with all their clients dead before they released the connection,
so let's track them to log it and remove them from clients
rinpatch [Thu, 7 May 2020 13:11:48 +0000 (16:11 +0300)]
Connection pool: Fix a possible infinite recursion if the pool is exhausted
rinpatch [Wed, 15 Jul 2020 12:26:25 +0000 (15:26 +0300)]
Connection pool: implement logging and telemetry events
rinpatch [Wed, 15 Jul 2020 12:24:47 +0000 (15:24 +0300)]
Connection Pool: register workers using :via
rinpatch [Wed, 6 May 2020 20:14:24 +0000 (23:14 +0300)]
Connection pool: Fix race conditions in limit enforcement
Fixes race conditions in limit enforcement by putting worker processes
in a DynamicSupervisor
rinpatch [Wed, 6 May 2020 18:51:10 +0000 (21:51 +0300)]
Connection pool: check that there actually is a result
Sometimes connections died before being released to the pool, resulting
in MatchErrors
rinpatch [Wed, 6 May 2020 18:41:34 +0000 (21:41 +0300)]
Connection Pool: don't enforce pool limits if no new connection needs to be opened
rinpatch [Tue, 5 May 2020 22:51:10 +0000 (01:51 +0300)]
Refactor gun pooling and simplify adapter option insertion
This patch refactors gun pooling to use Elixir process registry and
simplifies adapter option insertion.
Having the pool use process registry instead of a GenServer has a number of advantages:
- Simpler code: the initial implementation adds about half the lines of code it deletes
- Concurrency: unlike a GenServer, ETS-based registry can handle multiple checkout/checkin
requests at the same time
- Precise and easy idle connection clousure: current proposal for closing idle connections in
the GenServer-based pool needs to filter through all connections once a minute and compare their
last active time with closing time. With Elixir process registry this can be done
by just using `Process.send_after`/`Process.cancel_timer` in the worker process.
- Lower memory footprint: In my tests `gun-memory-leak` branch uses about 290mb on peak load (250 connections)
and 235mb on idle (5-10 connections). Registry-based pool uses 210mb on idle and 240mb on peak load
lain [Wed, 15 Jul 2020 10:12:44 +0000 (10:12 +0000)]
Merge branch 'tests/apc2s-update_outbox' into 'develop'
AP C2S tests: Make sure you can't use another user's AP id
See merge request pleroma/pleroma!2767
Haelwenn (lanodan) Monnier [Wed, 15 Jul 2020 07:58:35 +0000 (09:58 +0200)]
AP C2S tests: Make sure you can't use another user's AP id
Maksim Pechnikov [Tue, 14 Jul 2020 18:44:08 +0000 (21:44 +0300)]
Include port in host for signatures
feld [Tue, 14 Jul 2020 18:07:44 +0000 (18:07 +0000)]
Merge branch 'feature/1922-media-proxy-whitelist' into 'develop'
Support for hosts with scheme in MediaProxy whitelist setting
Closes #1922
See merge request pleroma/pleroma!2754
feld [Tue, 14 Jul 2020 18:05:36 +0000 (18:05 +0000)]
Merge branch 'fix/scrubber-recompile-warnings-as-errors' into 'develop'
Fix in-db configuration in dev environment
See merge request pleroma/pleroma!2764
feld [Tue, 14 Jul 2020 16:48:26 +0000 (16:48 +0000)]
Merge branch '1940-admin-token-oauthless-auth' into 'develop'
[#1940] Reinstated OAuth-less `admin_token` authentication
Closes #1940
See merge request pleroma/pleroma!2760
Ivan Tashkinov [Tue, 14 Jul 2020 16:31:05 +0000 (19:31 +0300)]
[#1940] Added `admin_token` param (as `admin_api_params/0`) to existing Admin API OpenAPI operations.
Mark Felder [Tue, 14 Jul 2020 16:24:58 +0000 (11:24 -0500)]
Clarify description and suggestion
feld [Tue, 14 Jul 2020 15:17:25 +0000 (15:17 +0000)]
Merge branch '1916-transmogrifier-skipping-oddities-in-address-fields' into 'develop'
Transmogrifier: filtering weirdness in address fields
Closes #1916
See merge request pleroma/pleroma!2762
rinpatch [Tue, 14 Jul 2020 10:39:08 +0000 (10:39 +0000)]
Merge branch 'fix/oban-2.0-in-db-config' into 'develop'
Migrate in-db config after updating to Oban 2.0
See merge request pleroma/pleroma!2765
rinpatch [Tue, 14 Jul 2020 08:41:30 +0000 (11:41 +0300)]
Fix in-db configuration in dev environment
Previously, in-db configuration only worked when `warnings_as_errors`
was disabled because re-compiling scrubbers on application restart
created a warning about module conflicts. This patch fixes that
by enabling `ignore_module_conflict` option of the compiler at runtime,
and enables `warnings_as_errors` in prod since there is no reason
to keep it disabled anymore.
rinpatch [Tue, 14 Jul 2020 09:00:53 +0000 (12:00 +0300)]
Migrate in-db config after updating to Oban 2.0
Haelwenn (lanodan) Monnier [Mon, 13 Jul 2020 13:47:13 +0000 (15:47 +0200)]
MRF Policies: Return a {:reject, reason} instead of {:reject, nil}
Ivan Tashkinov [Tue, 14 Jul 2020 08:58:41 +0000 (11:58 +0300)]
[#1940] Applied rate limit for requests with bad `admin_token`. Added doc warnings on `admin_token` setting.
Ivan Tashkinov [Sun, 19 Jul 2020 18:35:57 +0000 (21:35 +0300)]
[#1940] Reinstated OAuth-less `admin_token` authentication. Refactored UserIsAdminPlug (freed from checking admin scopes presence).
= [Mon, 13 Jul 2020 19:19:13 +0000 (22:19 +0300)]
Transmogrifier: filtering weirdness in address fields
Mark Felder [Mon, 13 Jul 2020 18:32:21 +0000 (13:32 -0500)]
Merge branch 'develop' into refactor/notification_settings
feld [Mon, 13 Jul 2020 15:45:25 +0000 (15:45 +0000)]
Merge branch 'config-behaviours-runtime' into 'develop'
Config/Docs: Expand behaviour suggestions modules at runtime
Closes #1941
See merge request pleroma/pleroma!2755
feld [Mon, 13 Jul 2020 15:43:46 +0000 (15:43 +0000)]
Merge branch 'features/profile-fields-emojo' into 'develop'
user: Add support for custom emojis in profile fields
See merge request pleroma/pleroma!2741
feld [Mon, 13 Jul 2020 15:42:20 +0000 (15:42 +0000)]
Merge branch 'chore/improve-database-config-docs' into 'develop'
Improve database config migration and add documentation
See merge request pleroma/pleroma!2756
feld [Mon, 13 Jul 2020 15:38:56 +0000 (15:38 +0000)]
Merge branch 'fix/mediaproxy-cache-settings-types' into 'develop'
Update types for :headers and :options settings in Pleroma.Web.MediaProxy.Invalidation.Http
See merge request pleroma/pleroma!2735
Mark Felder [Mon, 13 Jul 2020 15:32:17 +0000 (10:32 -0500)]
Lint
Mark Felder [Mon, 13 Jul 2020 14:56:05 +0000 (09:56 -0500)]
Show examples for both OTP and source
Mark Felder [Mon, 13 Jul 2020 13:39:56 +0000 (08:39 -0500)]
Pick up env for both source and OTP installs
Haelwenn [Mon, 13 Jul 2020 12:06:43 +0000 (12:06 +0000)]
Merge branch '1937-renaming' into 'develop'
ActivityPub: Don't rename a clashing nickname with the same ap id.
Closes #1937
See merge request pleroma/pleroma!2748
Mark Felder [Sun, 12 Jul 2020 15:59:12 +0000 (10:59 -0500)]
Still allow passing the arg, but fallback to MIX_ENV
Mark Felder [Sun, 12 Jul 2020 15:43:24 +0000 (10:43 -0500)]
Make the query more precise
Mark Felder [Sun, 12 Jul 2020 15:38:07 +0000 (10:38 -0500)]
Improve database config migration and add documentation
href [Sun, 12 Jul 2020 15:23:33 +0000 (17:23 +0200)]
Config/Docs: Expand behaviour suggestions at runtime
Alexander Strizhakov [Sat, 11 Jul 2020 07:36:36 +0000 (10:36 +0300)]
MediaProxy whitelist setting now supports hosts with scheme
added deprecation warning about using bare domains
Angelina Filippova [Sat, 11 Jul 2020 22:38:16 +0000 (01:38 +0300)]
Update types in MRF Keyword group
lain [Sat, 11 Jul 2020 11:31:44 +0000 (11:31 +0000)]
Merge branch 'feature/upload-filter-exiftool' into 'develop'
Add Pleroma.Upload.Filter.Exiftool
Closes #1935
See merge request pleroma/pleroma!2744
lain [Sat, 11 Jul 2020 11:30:02 +0000 (11:30 +0000)]
Merge branch 'patch-3' into 'develop'
docs: API: fix update_credentials endpoints path, clarify update/verify_credentials endpoints paths
See merge request pleroma/pleroma!2752
Alibek Omarov [Sat, 11 Jul 2020 11:02:13 +0000 (11:02 +0000)]
docs: API: fix update_credentials endpoints path, clarify update/verify_credentials endpoints paths
Angelina Filippova [Sat, 11 Jul 2020 01:32:44 +0000 (04:32 +0300)]
Remove :regex from types
Angelina Filippova [Sat, 11 Jul 2020 01:17:21 +0000 (04:17 +0300)]
Update types for :params, :match_actor and :replace settings
Mark Felder [Fri, 10 Jul 2020 22:08:54 +0000 (17:08 -0500)]
Remove unused @types
Mark Felder [Fri, 10 Jul 2020 22:07:28 +0000 (17:07 -0500)]
Add a moduledoc
Mark Felder [Fri, 10 Jul 2020 21:46:26 +0000 (16:46 -0500)]
Support Exiftool for stripping EXIF data
We really only want to strip location data anyway, and mogrify strips color profiles.
feld [Fri, 10 Jul 2020 17:39:46 +0000 (17:39 +0000)]
Merge branch 'issue/1790-updated-oban' into 'develop'
[#1790] updated oban to 2.0.0
See merge request pleroma/pleroma!2674
feld [Fri, 10 Jul 2020 16:42:35 +0000 (16:42 +0000)]
Merge branch 'fix/1924-s3-descriptions' into 'develop'
docs and descriptions for s3 settings
Closes #1924
See merge request pleroma/pleroma!2745
feld [Fri, 10 Jul 2020 16:38:05 +0000 (16:38 +0000)]
Merge branch 'chore/should-document-that' into 'develop'
Probably worth documenting the MediaProxy fix
See merge request pleroma/pleroma!2750
Mark Felder [Fri, 10 Jul 2020 16:34:53 +0000 (11:34 -0500)]
Merge branch 'develop' into issue/1790-updated-oban