From: kaniini Date: Sun, 10 Feb 2019 20:54:21 +0000 (+0000) Subject: Merge branch 'fix/credo-issues' into 'develop' X-Git-Url: http://git.squeep.com/?a=commitdiff_plain;h=6c8d15da110e86f799052c82df8b7b2404f8f722;hp=6ca633ddd30e8330e47f6456fe16fa72506e2e13;p=akkoma Merge branch 'fix/credo-issues' into 'develop' Fix credo issues See merge request pleroma/pleroma!786 --- diff --git a/README.md b/README.md index d39731ef4..4f22445d0 100644 --- a/README.md +++ b/README.md @@ -8,20 +8,7 @@ Pleroma is written in Elixir, high-performance and can run on small devices like For clients it supports both the [GNU Social API with Qvitter extensions](https://twitter-api.readthedocs.io/en/latest/index.html) and the [Mastodon client API](https://github.com/tootsuite/documentation/blob/master/Using-the-API/API.md). -Client applications that are committed to supporting Pleroma: - -* Mastalab (Android, Streaming Ready) -* nekonium (Android, Streaming Ready) -* Tusky (Android, No Streaming) -* Twidere (Android, No Streaming) -* Mast (iOS, Paid) -* Amaroq (iOS, No Streaming) - -Client applications that are known to work well: - -* Tootdon (Android + iOS, No Streaming) -* Tootle (iOS, No Streaming) -* Whalebird (Windows + Mac + Linux) +- [Client Applications for Pleroma](docs/Clients.md) No release has been made yet, but several servers have been online for months already. If you want to run your own server, feel free to contact us at @lain@pleroma.soykaf.com or in our dev chat at #pleroma on freenode or via matrix at . diff --git a/config/config.exs b/config/config.exs index c272ef34a..5db0ea9aa 100644 --- a/config/config.exs +++ b/config/config.exs @@ -238,6 +238,11 @@ config :pleroma, :mrf_simple, reject: [], accept: [] +config :pleroma, :mrf_keyword, + reject: [], + federated_timeline_removal: [], + replace: [] + config :pleroma, :rich_media, enabled: true config :pleroma, :media_proxy, diff --git a/docs/Clients.md b/docs/Clients.md new file mode 100644 index 000000000..28f8afafd --- /dev/null +++ b/docs/Clients.md @@ -0,0 +1,83 @@ +# Pleroma Clients +Note: Additionnal clients may be working but theses are officially supporting Pleroma. +Feel free to contact us to be added to this list! + +## Desktop +### Roma for Desktop +- Homepage: +- Source Code: ??? +- Platforms: Windows, Mac, (Linux?) + +### Social +- Source Code: +- Contact: [@brainblasted@social.libre.fi](https://social.libre.fi/users/brainblasted) +- Platforms: Linux (GNOME) +- Note(2019-01-28): Not at a pre-alpha stage yet + +### Whalebird +- Homepage: +- Source Code: +- Contact: [@h3poteto@pleroma.io](https://pleroma.io/users/h3poteto) +- Platforms: Windows, Mac, Linux + +## Handheld +### Amaroq +- Homepage: +- Source Code: +- Contact: [@eurasierboy@mastodon.social](https://mastodon.social/users/eurasierboy) +- Platforms: iOS + +### Nekonium +- Homepage: [F-Droid Repository](https://repo.gdgd.jp.net/), [Google Play](https://play.google.com/store/apps/details?id=com.apps.nekonium), [Amazon](https://www.amazon.co.jp/dp/B076FXPRBC/) +- Source: +- Contact: [@lin@pleroma.gdgd.jp.net](https://pleroma.gdgd.jp.net/users/lin) +- Platforms: Android + +### Mastalab +- Source Code: +- Contact: [@tom79@mastodon.social](https://mastodon.social/users/tom79) +- Platforms: Android + +### Roma +- Homepage: +- Source Code: ??? +- Platforms: iOS, Android + +### Tootdon +- Homepage: , +- Source Code: ??? +- Contact: [@tootdon@mstdn.jp](https://mstdn.jp/users/tootdon) +- Platforms: Android, iOS + +### Tusky +- Homepage: +- Source Code: +- Contact: [@ConnyDuck@mastodon.social](https://mastodon.social/users/ConnyDuck) +- Platforms: Android + +### Twidere +- Homepage: +- Source Code: , +- Contact: +- Platform: Android, iOS + +## Alternative Web Interfaces +### Brutaldon +- Homepage: +- Source Code: +- Contact: [@gcupc@glitch.social](https://glitch.social/users/gcupc) + +### Halcyon +- Source Code: +- Contact: [@halcyon@social.csswg.org](https://social.csswg.org/users/halcyon) + +### Pinafore +- Homepage: +- Source Code: +- Contact: [@pinafore@mastodon.technology](https://mastodon.technology/users/pinafore) +- Note: Pleroma support is a secondary goal + +### Sengi +- Source Code: +- Contact: [@sengi_app@mastodon.social](https://mastodon.social/users/sengi_app) +- Note(2019-01-28): The development is currently in a early stage. diff --git a/docs/config.md b/docs/config.md index ba6807760..74badd0da 100644 --- a/docs/config.md +++ b/docs/config.md @@ -171,6 +171,11 @@ This section is used to configure Pleroma-FE, unless ``:managed_config`` in ``:i * `delist_threshold`: Number of mentioned users after which the message gets delisted (the message can still be seen, but it will not show up in public timelines and mentioned users won't get notifications about it). Set to 0 to disable. * `reject_threshold`: Number of mentioned users after which the messaged gets rejected. Set to 0 to disable. +## :mrf_keyword +* `reject`: A list of patterns which result in message being rejected, each pattern can be a string or a [regular expression](https://hexdocs.pm/elixir/Regex.html) +* `federated_timeline_removal`: A list of patterns which result in message being removed from federated timelines (a.k.a unlisted), each pattern can be a string or a [regular expression](https://hexdocs.pm/elixir/Regex.html) +* `replace`: A list of tuples containing `{pattern, replacement}`, `pattern` can be a string or a [regular expression](https://hexdocs.pm/elixir/Regex.html) + ## :media_proxy * `enabled`: Enables proxying of remote media to the instance’s proxy * `base_url`: The base URL to access a user-uploaded file. Useful when you want to proxy the media files via another host/CDN fronts. diff --git a/installation/caddyfile-pleroma.example b/installation/caddyfile-pleroma.example index 03ff000b6..fcf76718e 100644 --- a/installation/caddyfile-pleroma.example +++ b/installation/caddyfile-pleroma.example @@ -23,6 +23,11 @@ example.tld { # If you do not want to use the mediaproxy function, remove these lines. # To use this directive, you need the http.cache plugin for Caddy. + cache { + match_path /media + default_max_age 720m + } + cache { match_path /proxy default_max_age 720m diff --git a/installation/init.d/pleroma b/installation/init.d/pleroma index 2b211df65..ed50bb551 100755 --- a/installation/init.d/pleroma +++ b/installation/init.d/pleroma @@ -1,7 +1,7 @@ #!/sbin/openrc-run # Requires OpenRC >= 0.35 -directory=~pleroma/pleroma +directory=/opt/pleroma command=/usr/bin/mix command_args="phx.server" @@ -18,4 +18,4 @@ pidfile="/var/run/pleroma.pid" depend() { need nginx postgresql -} \ No newline at end of file +} diff --git a/installation/pleroma.nginx b/installation/pleroma.nginx index a24bb0e61..a3d55e4bf 100644 --- a/installation/pleroma.nginx +++ b/installation/pleroma.nginx @@ -15,12 +15,13 @@ server { return 301 https://$server_name$request_uri; # Uncomment this if you need to use the 'webroot' method with certbot. Make sure - # that you also create the .well-known/acme-challenge directory structure in pleroma/priv/static and - # that is is accessible by the webserver. You may need to load this file with the ssl - # server block commented out, run certbot to get the certificate, and then uncomment it. + # that the directory exists and that it is accessible by the webserver. If you followed + # the guide, you already ran 'sudo mkdir -p /var/lib/letsencrypt' to create the folder. + # You may need to load this file with the ssl server block commented out, run certbot + # to get the certificate, and then uncomment it. # # location ~ /\.well-known/acme-challenge { - # root /pleroma/priv/static/; + # root /var/lib/letsencrypt/.well-known/acme-challenge; # } } diff --git a/installation/pleroma.service b/installation/pleroma.service index 72090bbc7..5dcbc1387 100644 --- a/installation/pleroma.service +++ b/installation/pleroma.service @@ -14,15 +14,17 @@ Environment="MIX_ENV=prod" ; Make sure that all paths fit your installation. ; Path to the home directory of the user running the Pleroma service. -Environment="HOME=/home/pleroma" +Environment="HOME=/var/lib/pleroma" ; Path to the folder containing the Pleroma installation. -WorkingDirectory=/home/pleroma/pleroma +WorkingDirectory=/opt/pleroma ; Path to the Mix binary. ExecStart=/usr/bin/mix phx.server ; Some security directives. ; Use private /tmp and /var/tmp folders inside a new file system namespace, which are discarded after the process stops. PrivateTmp=true +; The /home, /root, and /run/user folders can not be accessed by this service anymore. If your Pleroma user has its home folder in one of the restricted places, or use one of these folders as its working directory, you have to set this to false. +ProtectHome=true ; Mount /usr, /boot, and /etc as read-only for processes invoked by this service. ProtectSystem=full ; Sets up a new /dev mount for the process and only adds API pseudo devices like /dev/null, /dev/zero or /dev/random but not physical devices. Disabled by default because it may not work on devices like the Raspberry Pi. diff --git a/lib/pleroma/object.ex b/lib/pleroma/object.ex index 5f1fc801b..dabb49536 100644 --- a/lib/pleroma/object.ex +++ b/lib/pleroma/object.ex @@ -20,9 +20,29 @@ defmodule Pleroma.Object do timestamps() end + def insert_or_get(cng) do + {_, data} = fetch_field(cng, :data) + id = data["id"] || data[:id] + key = "object:#{id}" + + fetcher = fn _ -> + with nil <- get_by_ap_id(id), + {:ok, object} <- Repo.insert(cng) do + {:commit, object} + else + %Object{} = object -> {:commit, object} + e -> {:ignore, e} + end + end + + with {state, object} when state in [:commit, :ok] <- Cachex.fetch(:object_cache, key, fetcher) do + {:ok, object} + end + end + def create(data) do Object.change(%Object{}, %{data: data}) - |> Repo.insert() + |> insert_or_get() end def change(struct, params \\ %{}) do diff --git a/lib/pleroma/user.ex b/lib/pleroma/user.ex index b44ba1279..0060d966b 100644 --- a/lib/pleroma/user.ex +++ b/lib/pleroma/user.ex @@ -106,12 +106,6 @@ defmodule Pleroma.User do "#{ap_id(user)}/followers" end - def follow_changeset(struct, params \\ %{}) do - struct - |> cast(params, [:following]) - |> validate_required([:following]) - end - def user_info(%User{} = user) do oneself = if user.local, do: 1, else: 0 @@ -266,8 +260,8 @@ defmodule Pleroma.User do @doc "Inserts provided changeset, performs post-registration actions (confirmation email sending etc.)" def register(%Ecto.Changeset{} = changeset) do with {:ok, user} <- Repo.insert(changeset), - {:ok, _} <- try_send_confirmation_email(user), - {:ok, user} <- autofollow_users(user) do + {:ok, user} <- autofollow_users(user), + {:ok, _} <- try_send_confirmation_email(user) do {:ok, user} end end @@ -317,10 +311,13 @@ defmodule Pleroma.User do end end - @doc "A mass follow for local users. Ignores blocks and has no side effects" + @doc "A mass follow for local users. Respects blocks but does not create activities." @spec follow_all(User.t(), list(User.t())) :: {atom(), User.t()} def follow_all(follower, followeds) do - followed_addresses = Enum.map(followeds, fn %{follower_address: fa} -> fa end) + followed_addresses = + followeds + |> Enum.reject(fn %{ap_id: ap_id} -> ap_id in follower.info.blocks end) + |> Enum.map(fn %{follower_address: fa} -> fa end) q = from(u in User, diff --git a/lib/pleroma/web/activity_pub/mrf/keyword_policy.ex b/lib/pleroma/web/activity_pub/mrf/keyword_policy.ex new file mode 100644 index 000000000..ce6d2e529 --- /dev/null +++ b/lib/pleroma/web/activity_pub/mrf/keyword_policy.ex @@ -0,0 +1,73 @@ +# Pleroma: A lightweight social networking server +# Copyright © 2017-2019 Pleroma Authors +# SPDX-License-Identifier: AGPL-3.0-only + +defmodule Pleroma.Web.ActivityPub.MRF.KeywordPolicy do + @behaviour Pleroma.Web.ActivityPub.MRF + defp string_matches?(string, pattern) when is_binary(pattern) do + String.contains?(string, pattern) + end + + defp string_matches?(string, pattern) do + String.match?(string, pattern) + end + + defp check_reject(%{"object" => %{"content" => content}} = message) do + if Enum.any?(Pleroma.Config.get([:mrf_keyword, :reject]), fn pattern -> + string_matches?(content, pattern) + end) do + {:reject, nil} + else + {:ok, message} + end + end + + defp check_ftl_removal(%{"to" => to, "object" => %{"content" => content}} = message) do + if "https://www.w3.org/ns/activitystreams#Public" in to and + Enum.any?(Pleroma.Config.get([:mrf_keyword, :federated_timeline_removal]), fn pattern -> + string_matches?(content, pattern) + end) do + to = List.delete(to, "https://www.w3.org/ns/activitystreams#Public") + cc = ["https://www.w3.org/ns/activitystreams#Public" | message["cc"] || []] + + message = + message + |> Map.put("to", to) + |> Map.put("cc", cc) + + {:ok, message} + else + {:ok, message} + end + end + + defp check_replace(%{"object" => %{"content" => content}} = message) do + content = + Enum.reduce(Pleroma.Config.get([:mrf_keyword, :replace]), content, fn {pattern, replacement}, + acc -> + String.replace(acc, pattern, replacement) + end) + + {:ok, put_in(message["object"]["content"], content)} + end + + @impl true + def filter(%{"object" => %{"content" => nil}} = message) do + {:ok, message} + end + + @impl true + def filter(%{"type" => "Create", "object" => %{"content" => _content}} = message) do + with {:ok, message} <- check_reject(message), + {:ok, message} <- check_ftl_removal(message), + {:ok, message} <- check_replace(message) do + {:ok, message} + else + _e -> + {:reject, nil} + end + end + + @impl true + def filter(message), do: {:ok, message} +end diff --git a/lib/pleroma/web/activity_pub/utils.ex b/lib/pleroma/web/activity_pub/utils.ex index 964e11c9d..da6cca4dd 100644 --- a/lib/pleroma/web/activity_pub/utils.ex +++ b/lib/pleroma/web/activity_pub/utils.ex @@ -142,14 +142,8 @@ defmodule Pleroma.Web.ActivityPub.Utils do context = context || generate_id("contexts") changeset = Object.context_mapping(context) - case Repo.insert(changeset) do - {:ok, object} -> - object - - # This should be solved by an upsert, but it seems ecto - # has problems accessing the constraint inside the jsonb. - {:error, _} -> - Object.get_cached_by_ap_id(context) + with {:ok, object} <- Object.insert_or_get(changeset) do + object end end diff --git a/lib/pleroma/web/nodeinfo/nodeinfo_controller.ex b/lib/pleroma/web/nodeinfo/nodeinfo_controller.ex index c38827165..f4867d05b 100644 --- a/lib/pleroma/web/nodeinfo/nodeinfo_controller.ex +++ b/lib/pleroma/web/nodeinfo/nodeinfo_controller.ex @@ -45,6 +45,33 @@ defmodule Pleroma.Web.Nodeinfo.NodeinfoController do Application.get_env(:pleroma, :mrf_simple) |> Enum.into(%{}) + # This horror is needed to convert regex sigils to strings + mrf_keyword = + Application.get_env(:pleroma, :mrf_keyword, []) + |> Enum.map(fn {key, value} -> + {key, + Enum.map(value, fn + {pattern, replacement} -> + %{ + "pattern" => + if not is_binary(pattern) do + inspect(pattern) + else + pattern + end, + "replacement" => replacement + } + + pattern -> + if not is_binary(pattern) do + inspect(pattern) + else + pattern + end + end)} + end) + |> Enum.into(%{}) + mrf_policies = MRF.get_policies() |> Enum.map(fn policy -> to_string(policy) |> String.split(".") |> List.last() end) @@ -72,6 +99,7 @@ defmodule Pleroma.Web.Nodeinfo.NodeinfoController do %{ mrf_policies: mrf_policies, mrf_simple: mrf_simple, + mrf_keyword: mrf_keyword, mrf_user_allowlist: mrf_user_allowlist, quarantined_instances: quarantined } diff --git a/lib/pleroma/web/twitter_api/twitter_api.ex b/lib/pleroma/web/twitter_api/twitter_api.ex index db521a3ad..162beb9be 100644 --- a/lib/pleroma/web/twitter_api/twitter_api.ex +++ b/lib/pleroma/web/twitter_api/twitter_api.ex @@ -310,16 +310,8 @@ defmodule Pleroma.Web.TwitterAPI.TwitterAPI do else _e -> changeset = Object.context_mapping(context) - - case Repo.insert(changeset) do - {:ok, %{id: id}} -> - id - - # This should be solved by an upsert, but it seems ecto - # has problems accessing the constraint inside the jsonb. - {:error, _} -> - Object.get_cached_by_ap_id(context).id - end + {:ok, object} = Object.insert_or_get(changeset) + object.id end end diff --git a/test/object_test.exs b/test/object_test.exs index 72194975d..ab6431012 100644 --- a/test/object_test.exs +++ b/test/object_test.exs @@ -57,4 +57,32 @@ defmodule Pleroma.ObjectTest do assert cached_object.data["type"] == "Tombstone" end end + + describe "insert_or_get" do + test "inserting the same object twice (by id) just returns the original object" do + data = %{data: %{"id" => Ecto.UUID.generate()}} + cng = Object.change(%Object{}, data) + {:ok, object} = Object.insert_or_get(cng) + {:ok, second_object} = Object.insert_or_get(cng) + + Cachex.clear(:object_cache) + {:ok, third_object} = Object.insert_or_get(cng) + + assert object == second_object + assert object == third_object + end + end + + describe "create" do + test "inserts an object for a given data set" do + data = %{"id" => Ecto.UUID.generate()} + + {:ok, object} = Object.create(data) + assert object.data["id"] == data["id"] + + # Works when doing it twice. + {:ok, object} = Object.create(data) + assert object.data["id"] == data["id"] + end + end end diff --git a/test/user_test.exs b/test/user_test.exs index 98d3bc464..523ab1ea4 100644 --- a/test/user_test.exs +++ b/test/user_test.exs @@ -53,16 +53,20 @@ defmodule Pleroma.UserTest do followed_zero = insert(:user) followed_one = insert(:user) followed_two = insert(:user) + blocked = insert(:user) not_followed = insert(:user) + {:ok, user} = User.block(user, blocked) + {:ok, user} = User.follow(user, followed_zero) - {:ok, user} = User.follow_all(user, [followed_one, followed_two]) + {:ok, user} = User.follow_all(user, [followed_one, followed_two, blocked]) assert User.following?(user, followed_one) assert User.following?(user, followed_two) assert User.following?(user, followed_zero) refute User.following?(user, not_followed) + refute User.following?(user, blocked) end test "follow_all follows mutliple users without duplicating" do diff --git a/test/web/activity_pub/mrf/keyword_policy_test.exs b/test/web/activity_pub/mrf/keyword_policy_test.exs new file mode 100644 index 000000000..67a5858d7 --- /dev/null +++ b/test/web/activity_pub/mrf/keyword_policy_test.exs @@ -0,0 +1,111 @@ +# Pleroma: A lightweight social networking server +# Copyright © 2019 Pleroma Authors +# SPDX-License-Identifier: AGPL-3.0-only + +defmodule Pleroma.Web.ActivityPub.MRF.KeywordPolicyTest do + use Pleroma.DataCase + + alias Pleroma.Web.ActivityPub.MRF.KeywordPolicy + + setup do + Pleroma.Config.put([:mrf_keyword], %{reject: [], federated_timeline_removal: [], replace: []}) + end + + describe "rejecting based on keywords" do + test "rejects if string matches" do + Pleroma.Config.put([:mrf_keyword, :reject], ["pun"]) + + message = %{ + "type" => "Create", + "object" => %{"content" => "just a daily reminder that compLAINer is a good pun"} + } + + assert {:reject, nil} == KeywordPolicy.filter(message) + end + + test "rejects if regex matches" do + Pleroma.Config.put([:mrf_keyword, :reject], [~r/comp[lL][aA][iI][nN]er/]) + + assert true == + Enum.all?(["complainer", "compLainer", "compLAiNer", "compLAINer"], fn content -> + message = %{ + "type" => "Create", + "object" => %{ + "content" => "just a daily reminder that #{content} is a good pun" + } + } + + {:reject, nil} == KeywordPolicy.filter(message) + end) + end + end + + describe "delisting from ftl based on keywords" do + test "delists if string matches" do + Pleroma.Config.put([:mrf_keyword, :federated_timeline_removal], ["pun"]) + + message = %{ + "to" => ["https://www.w3.org/ns/activitystreams#Public"], + "type" => "Create", + "object" => %{"content" => "just a daily reminder that compLAINer is a good pun"} + } + + {:ok, result} = KeywordPolicy.filter(message) + assert ["https://www.w3.org/ns/activitystreams#Public"] == result["cc"] + refute ["https://www.w3.org/ns/activitystreams#Public"] == result["to"] + end + + test "delists if regex matches" do + Pleroma.Config.put([:mrf_keyword, :federated_timeline_removal], [~r/comp[lL][aA][iI][nN]er/]) + + assert true == + Enum.all?(["complainer", "compLainer", "compLAiNer", "compLAINer"], fn content -> + message = %{ + "type" => "Create", + "to" => ["https://www.w3.org/ns/activitystreams#Public"], + "object" => %{ + "content" => "just a daily reminder that #{content} is a good pun" + } + } + + {:ok, result} = KeywordPolicy.filter(message) + + ["https://www.w3.org/ns/activitystreams#Public"] == result["cc"] and + not (["https://www.w3.org/ns/activitystreams#Public"] == result["to"]) + end) + end + end + + describe "replacing keywords" do + test "replaces keyword if string matches" do + Pleroma.Config.put([:mrf_keyword, :replace], [{"opensource", "free software"}]) + + message = %{ + "type" => "Create", + "to" => ["https://www.w3.org/ns/activitystreams#Public"], + "object" => %{"content" => "ZFS is opensource"} + } + + {:ok, %{"object" => %{"content" => result}}} = KeywordPolicy.filter(message) + assert result == "ZFS is free software" + end + + test "replaces keyword if regex matches" do + Pleroma.Config.put([:mrf_keyword, :replace], [ + {~r/open(-|\s)?source\s?(software)?/, "free software"} + ]) + + assert true == + Enum.all?(["opensource", "open-source", "open source"], fn content -> + message = %{ + "type" => "Create", + "to" => ["https://www.w3.org/ns/activitystreams#Public"], + "object" => %{"content" => "ZFS is #{content}"} + } + + {:ok, %{"object" => %{"content" => result}}} = KeywordPolicy.filter(message) + result == "ZFS is free software" + end) + end + end +end