Merge branch 'develop' into feature/addressable-lists
authorEgor Kislitsyn <egor@kislitsyn.com>
Thu, 11 Jul 2019 06:26:59 +0000 (13:26 +0700)
committerEgor Kislitsyn <egor@kislitsyn.com>
Thu, 11 Jul 2019 06:26:59 +0000 (13:26 +0700)
1  2 
CHANGELOG.md
docs/api/differences_in_mastoapi_responses.md
lib/pleroma/web/activity_pub/activity_pub.ex
lib/pleroma/web/activity_pub/publisher.ex
lib/pleroma/web/activity_pub/transmogrifier.ex
lib/pleroma/web/common_api/common_api.ex
lib/pleroma/web/common_api/utils.ex
lib/pleroma/web/salmon/salmon.ex
test/web/activity_pub/activity_pub_test.exs
test/web/activity_pub/transmogrifier_test.exs
test/web/common_api/common_api_test.exs

diff --combined CHANGELOG.md
index e569c5ad2077bb4120195356de8c190b42697bad,f27446f36b954a63cbb71d13781edeb4970c5c92..ff0cb8740ead2c989fdaff28f619fa71f9b1c34d
@@@ -3,7 -3,37 +3,37 @@@ All notable changes to this project wil
  
  The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
  
- ## [unreleased]
+ ## [Unreleased]
+ ### Changed
+ - **Breaking:** Configuration: A setting to explicitly disable the mailer was added, defaulting to true, if you are using a mailer add `config :pleroma, Pleroma.Emails.Mailer, enabled: true` to your config
+ - Configuration: OpenGraph and TwitterCard providers enabled by default
+ - Configuration: Filter.AnonymizeFilename added ability to retain file extension with custom text
+ - NodeInfo: Return `skipThreadContainment` in `metadata` for the `skip_thread_containment` option
+ ### Fixed
+ - Not being able to pin unlisted posts
+ - Metadata rendering errors resulting in the entire page being inaccessible
+ - Mastodon API: Handling of search timeouts (`/api/v1/search` and `/api/v2/search`)
+ - Mastodon API: Embedded relationships not being properly rendered in the Account entity of Status entity
+ ### Added
+ - MRF: Support for priming the mediaproxy cache (`Pleroma.Web.ActivityPub.MRF.MediaProxyWarmingPolicy`)
+ Configuration: `federation_incoming_replies_max_depth` option
+ - Mastodon API: Support for the [`tagged` filter](https://github.com/tootsuite/mastodon/pull/9755) in [`GET /api/v1/accounts/:id/statuses`](https://docs.joinmastodon.org/api/rest/accounts/#get-api-v1-accounts-id-statuses)
+ - Mastodon API, streaming: Add support for passing the token in the `Sec-WebSocket-Protocol` header
+ - Mastodon API, extension: Ability to reset avatar, profile banner, and background
+ - Admin API: Return users' tags when querying reports
+ - Admin API: Return avatar and display name when querying users
+ - Admin API: Allow querying user by ID
+ - Added synchronization of following/followers counters for external users
+ - Configuration: `enabled` option for `Pleroma.Emails.Mailer`, defaulting to `false`.
+ - Mastodon API: Add support for categories for custom emojis by reusing the group feature. <https://github.com/tootsuite/mastodon/pull/11196>
+ ## [1.0.0] - 2019-06-29
+ ### Security
+ - Mastodon API: Fix display names not being sanitized
+ - Rich media: Do not crawl private IP ranges
  ### Added
  - Add a generic settings store for frontends / clients to use.
  - Explicit addressing option for posting.
@@@ -11,6 -41,7 +41,7 @@@
  - [MongooseIM](https://github.com/esl/MongooseIM) http authentication support.
  - LDAP authentication
  - External OAuth provider authentication
+ - Support for building a release using [`mix release`](https://hexdocs.pm/mix/master/Mix.Tasks.Release.html)
  - A [job queue](https://git.pleroma.social/pleroma/pleroma_job_queue) for federation, emails, web push, etc.
  - [Prometheus](https://prometheus.io/) metrics
  - Support for Mastodon's remote interaction
  - Mix Tasks: `mix pleroma.database remove_embedded_objects`
  - Mix Tasks: `mix pleroma.database update_users_following_followers_counts`
  - Mix Tasks: `mix pleroma.user toggle_confirmed`
+ - Mix Tasks: `mix pleroma.config migrate_to_db`
+ - Mix Tasks: `mix pleroma.config migrate_from_db`
  - Federation: Support for `Question` and `Answer` objects
  - Federation: Support for reports
  - Configuration: `poll_limits` option
+ - Configuration: `pack_extensions` option
  - Configuration: `safe_dm_mentions` option
  - Configuration: `link_name` option
  - Configuration: `fetch_initial_posts` option
  - Configuration: `notify_email` option
  - Configuration: Media proxy `whitelist` option
  - Configuration: `report_uri` option
+ - Configuration: `limit_to_local_content` option
  - Pleroma API: User subscriptions
  - Pleroma API: Healthcheck endpoint
  - Pleroma API: `/api/v1/pleroma/mascot` per-user frontend mascot configuration endpoints
  - Admin API: added filters (role, tags, email, name) for users endpoint
  - Admin API: Endpoints for managing reports
  - Admin API: Endpoints for deleting and changing the scope of individual reported statuses
+ - Admin API: Endpoints to view and change config settings.
  - AdminFE: initial release with basic user management accessible at /pleroma/admin/
+ - Mastodon API: Add chat token to `verify_credentials` response
+ - Mastodon API: Add background image setting to `update_credentials`
  - Mastodon API: [Scheduled statuses](https://docs.joinmastodon.org/api/rest/scheduled-statuses/)
  - Mastodon API: `/api/v1/notifications/destroy_multiple` (glitch-soc extension)
  - Mastodon API: `/api/v1/pleroma/accounts/:id/favourites` (API extension)
  - MRF: Support for rejecting reports from specific instances (`mrf_simple`)
  - MRF: Support for stripping avatars and banner images from specific instances (`mrf_simple`)
  - MRF: Support for running subchains.
 +- Addressable lists
  - Configuration: `skip_thread_containment` option
+ - Configuration: `rate_limit` option. See `Pleroma.Plugs.RateLimiter` documentation for details.
+ - MRF: Support for filtering out likely spam messages by rejecting posts from new users that contain links.
+ - Configuration: `ignore_hosts` option
+ - Configuration: `ignore_tld` option
+ - Configuration: default syslog tag "Pleroma" is now lowercased to "pleroma"
  
  ### Changed
+ - **Breaking:** bind to 127.0.0.1 instead of 0.0.0.0 by default
  - **Breaking:** Configuration: move from Pleroma.Mailer to Pleroma.Emails.Mailer
+ - Thread containment / test for complete visibility will be skipped by default.
  - Enforcement of OAuth scopes
  - Add multiple use/time expiring invite token
  - Restyled OAuth pages to fit with Pleroma's default theme
  - Federation: Expand the audience of delete activities to all recipients of the deleted object
  - Federation: Removed `inReplyToStatusId` from objects
  - Configuration: Dedupe enabled by default
+ - Configuration: Default log level in `prod` environment is now set to `warn`
  - Configuration: Added `extra_cookie_attrs` for setting non-standard cookie attributes. Defaults to ["SameSite=Lax"] so that remote follows work.
  - Timelines: Messages involving people you have blocked will be excluded from the timeline in all cases instead of just repeats.
  - Admin API: Move the user related API to `api/pleroma/admin/users`
  - Posts which are marked sensitive or tagged nsfw no longer have link previews.
  - HTTP connection timeout is now set to 10 seconds.
  - Respond with a 404 Not implemented JSON error message when requested API is not implemented
+ - Rich Media: crawl only https URLs.
  
  ### Fixed
+ - Follow requests don't get 'stuck' anymore.
  - Added an FTS index on objects. Running `vacuum analyze` and setting a larger `work_mem` is recommended.
  - Followers counter not being updated when a follower is blocked
  - Deactivated users being able to request an access token
index fb77ce68da3f3da38cf86aeb077fbca246f85d48,2cbe1458dfe768dc5b799cc3c0c93a2a4944884d..1de1e3b44f262935534cad0d5b237f7e5e92d9be
@@@ -44,6 -44,15 +44,15 @@@ Has these additional fields under the `
  - `hide_followers`: boolean, true when the user has follower hiding enabled
  - `hide_follows`: boolean, true when the user has follow hiding enabled
  - `settings_store`: A generic map of settings for frontends. Opaque to the backend. Only returned in `verify_credentials` and `update_credentials`
+ - `chat_token`: The token needed for Pleroma chat. Only returned in `verify_credentials`
+ ### Extensions for PleromaFE
+ These endpoints added for controlling PleromaFE features over the Mastodon API
+ - PATCH `/api/v1/accounts/update_avatar`: Set/clear user avatar image
+ - PATCH `/api/v1/accounts/update_banner`: Set/clear user banner image
+ - PATCH `/api/v1/accounts/update_background`: Set/clear user background image
  
  ### Source
  
@@@ -71,7 -80,6 +80,7 @@@ Additional parameters can be added to t
  - `preview`: boolean, if set to `true` the post won't be actually posted, but the status entitiy would still be rendered back. This could be useful for previewing rich text/custom emoji, for example.
  - `content_type`: string, contain the MIME type of the status, it is transformed into HTML by the backend. You can get the list of the supported MIME types with the nodeinfo endpoint.
  - `to`: A list of nicknames (like `lain@soykaf.club` or `lain` on the local server) that will be used to determine who is going to be addressed by this post. Using this will disable the implicit addressing by mentioned names in the `status` body, only the people in the `to` list will be addressed. The normal rules for for post visibility are not affected by this and will still apply.
 +- `visibility`: string, besides standard MastoAPI values (`direct`, `private`, `unlisted` or `public`) it can be used to address a List by setting it to `list:LIST_ID`.
  
  ## PATCH `/api/v1/update_credentials`
  
@@@ -85,6 -93,7 +94,7 @@@ Additional parameters can be added to t
  - `default_scope` - the scope returned under `privacy` key in Source subentity
  - `pleroma_settings_store` - Opaque user settings to be saved on the backend.
  - `skip_thread_containment` - if true, skip filtering out broken threads
+ - `pleroma_background_image` - sets the background image of the user.
  
  ### Pleroma Settings Store
  Pleroma has mechanism that allows frontends to save blobs of json for each user on the backend. This can be used to save frontend-specific settings for a user that the backend does not need to know about.
index 73c6e4cbf3d6d8b4617a000e578d8318bb099b4b,41b55bbabcf1ddfd24674a49d13b6b97aaa3939b..20f72e676237a3e378bd15100858c7c579247848
@@@ -26,16 -26,19 +26,16 @@@ defmodule Pleroma.Web.ActivityPub.Activ
    # For Announce activities, we filter the recipients based on following status for any actors
    # that match actual users.  See issue #164 for more information about why this is necessary.
    defp get_recipients(%{"type" => "Announce"} = data) do
 -    to = data["to"] || []
 -    cc = data["cc"] || []
 +    to = Map.get(data, "to", [])
 +    cc = Map.get(data, "cc", [])
 +    bcc = Map.get(data, "bcc", [])
      actor = User.get_cached_by_ap_id(data["actor"])
  
      recipients =
 -      (to ++ cc)
 -      |> Enum.filter(fn recipient ->
 +      Enum.filter(Enum.concat([to, cc, bcc]), fn recipient ->
          case User.get_cached_by_ap_id(recipient) do
 -          nil ->
 -            true
 -
 -          user ->
 -            User.following?(user, actor)
 +          nil -> true
 +          user -> User.following?(user, actor)
          end
        end)
  
    end
  
    defp get_recipients(%{"type" => "Create"} = data) do
 -    to = data["to"] || []
 -    cc = data["cc"] || []
 -    actor = data["actor"] || []
 -    recipients = (to ++ cc ++ [actor]) |> Enum.uniq()
 +    to = Map.get(data, "to", [])
 +    cc = Map.get(data, "cc", [])
 +    bcc = Map.get(data, "bcc", [])
 +    actor = Map.get(data, "actor", [])
 +    recipients = [to, cc, bcc, [actor]] |> Enum.concat() |> Enum.uniq()
      {recipients, to, cc}
    end
  
    defp get_recipients(data) do
 -    to = data["to"] || []
 -    cc = data["cc"] || []
 -    recipients = to ++ cc
 +    to = Map.get(data, "to", [])
 +    cc = Map.get(data, "cc", [])
 +    bcc = Map.get(data, "bcc", [])
 +    recipients = Enum.concat([to, cc, bcc])
      {recipients, to, cc}
    end
  
      end)
    end
  
+   def stream_out_participations(%Object{data: %{"context" => context}}, user) do
+     with %Conversation{} = conversation <- Conversation.get_for_ap_id(context),
+          conversation = Repo.preload(conversation, :participations),
+          last_activity_id =
+            fetch_latest_activity_id_for_context(conversation.ap_id, %{
+              "user" => user,
+              "blocking_user" => user
+            }) do
+       if last_activity_id do
+         stream_out_participations(conversation.participations)
+       end
+     end
+   end
+   def stream_out_participations(_, _), do: :noop
    def stream_out(activity) do
      public = "https://www.w3.org/ns/activitystreams#Public"
  
      end
    end
  
+   def delete(%User{ap_id: ap_id, follower_address: follower_address} = user) do
+     with data <- %{
+            "to" => [follower_address],
+            "type" => "Delete",
+            "actor" => ap_id,
+            "object" => %{"type" => "Person", "id" => ap_id}
+          },
+          {:ok, activity} <- insert(data, true, true),
+          :ok <- maybe_federate(activity) do
+       {:ok, user}
+     end
+   end
    def delete(%Object{data: %{"id" => id, "actor" => actor}} = object, local \\ true) do
      user = User.get_cached_by_ap_id(actor)
      to = (object.data["to"] || []) ++ (object.data["cc"] || [])
             "to" => to,
             "deleted_activity_id" => activity && activity.id
           },
-          {:ok, activity} <- insert(data, local),
+          {:ok, activity} <- insert(data, local, false),
+          stream_out_participations(object, user),
           _ <- decrease_replies_count_if_reply(object),
           # Changing note count prior to enqueuing federation task in order to avoid
           # race conditions on updating user.info
    defp maybe_order(query, _), do: query
  
    def fetch_activities_query(recipients, opts \\ %{}) do
 -    base_query = from(activity in Activity)
 -
      config = %{
        skip_thread_containment: Config.get([:instance, :skip_thread_containment])
      }
  
 -    base_query
 +    Activity
      |> maybe_preload_objects(opts)
      |> maybe_preload_bookmarks(opts)
      |> maybe_set_thread_muted_field(opts)
    end
  
    def fetch_activities(recipients, opts \\ %{}) do
 -    fetch_activities_query(recipients, opts)
 +    list_memberships = Pleroma.List.memberships(opts["user"])
 +
 +    fetch_activities_query(recipients ++ list_memberships, opts)
      |> Pagination.fetch_paginated(opts)
      |> Enum.reverse()
 +    |> maybe_update_cc(list_memberships, opts["user"])
 +  end
 +
 +  defp maybe_update_cc(activities, list_memberships, %User{ap_id: user_ap_id})
 +       when is_list(list_memberships) and length(list_memberships) > 0 do
 +    Enum.map(activities, fn
 +      %{data: %{"bcc" => bcc}} = activity when is_list(bcc) and length(bcc) > 0 ->
 +        if Enum.any?(bcc, &(&1 in list_memberships)) do
 +          update_in(activity.data["cc"], &[user_ap_id | &1])
 +        else
 +          activity
 +        end
 +
 +      activity ->
 +        activity
 +    end)
    end
  
 +  defp maybe_update_cc(activities, _, _), do: activities
 +
    def fetch_activities_bounded_query(query, recipients, recipients_with_public) do
      from(activity in query,
        where:
index f376e5618cad8c4d9bb73cd3e514af252341fb3c,a05e032639ed542893f88eb1fafa6b7b48885028..b7dc90caa554e4a68896c66170a01fedfe3a0cc5
@@@ -88,71 -88,22 +88,71 @@@ defmodule Pleroma.Web.ActivityPub.Publi
        true
      else
        inbox_info = URI.parse(inbox)
-       !Enum.member?(Pleroma.Config.get([:instance, :quarantined_instances], []), inbox_info.host)
+       !Enum.member?(Config.get([:instance, :quarantined_instances], []), inbox_info.host)
      end
    end
  
 -  @doc """
 -  Publishes an activity to all relevant peers.
 -  """
 -  def publish(%User{} = actor, %Activity{} = activity) do
 -    remote_followers =
 +  defp recipients(actor, activity) do
 +    followers =
        if actor.follower_address in activity.recipients do
          {:ok, followers} = User.get_followers(actor)
 -        followers |> Enum.filter(&(!&1.local))
 +        Enum.filter(followers, &(!&1.local))
        else
          []
        end
  
 +    Pleroma.Web.Salmon.remote_users(actor, activity) ++ followers
 +  end
 +
 +  defp get_cc_ap_ids(ap_id, recipients) do
 +    host = Map.get(URI.parse(ap_id), :host)
 +
 +    recipients
 +    |> Enum.filter(fn %User{ap_id: ap_id} -> Map.get(URI.parse(ap_id), :host) == host end)
 +    |> Enum.map(& &1.ap_id)
 +  end
 +
 +  @doc """
 +  Publishes an activity with BCC to all relevant peers.
 +  """
 +
 +  def publish(actor, %{data: %{"bcc" => bcc}} = activity) when is_list(bcc) and bcc != [] do
 +    public = is_public?(activity)
 +    {:ok, data} = Transmogrifier.prepare_outgoing(activity.data)
 +
 +    recipients = recipients(actor, activity)
 +
 +    recipients
 +    |> Enum.filter(&User.ap_enabled?/1)
 +    |> Enum.map(fn %{info: %{source_data: data}} -> data["inbox"] end)
 +    |> Enum.filter(fn inbox -> should_federate?(inbox, public) end)
 +    |> Instances.filter_reachable()
 +    |> Enum.each(fn {inbox, unreachable_since} ->
 +      %User{ap_id: ap_id} =
 +        Enum.find(recipients, fn %{info: %{source_data: data}} -> data["inbox"] == inbox end)
 +
 +      cc = get_cc_ap_ids(ap_id, recipients)
 +
 +      json =
 +        data
 +        |> Map.put("cc", cc)
 +        |> Map.put("directMessage", true)
 +        |> Jason.encode!()
 +
 +      Pleroma.Web.Federator.Publisher.enqueue_one(__MODULE__, %{
 +        inbox: inbox,
 +        json: json,
 +        actor: actor,
 +        id: activity.data["id"],
 +        unreachable_since: unreachable_since
 +      })
 +    end)
 +  end
 +
 +  @doc """
 +  Publishes an activity to all relevant peers.
 +  """
 +  def publish(%User{} = actor, %Activity{} = activity) do
      public = is_public?(activity)
  
      if public && Config.get([:instance, :allow_relay]) do
      {:ok, data} = Transmogrifier.prepare_outgoing(activity.data)
      json = Jason.encode!(data)
  
 -    (Pleroma.Web.Salmon.remote_users(activity) ++ remote_followers)
 +    recipients(actor, activity)
      |> Enum.filter(fn user -> User.ap_enabled?(user) end)
      |> Enum.map(fn %{info: %{source_data: data}} ->
        (is_map(data["endpoints"]) && Map.get(data["endpoints"], "sharedInbox")) || data["inbox"]
index d22d24479c8a51c8acd7aa558adbe953749f2b53,e34fe661100923c5a5f786078a1d194d76286bfe..ad741122ff59919f6863e0bd93421e1315a46a80
@@@ -14,6 -14,7 +14,7 @@@ defmodule Pleroma.Web.ActivityPub.Trans
    alias Pleroma.Web.ActivityPub.ActivityPub
    alias Pleroma.Web.ActivityPub.Utils
    alias Pleroma.Web.ActivityPub.Visibility
+   alias Pleroma.Web.Federator
  
    import Ecto.Query
  
    @doc """
    Modifies an incoming AP object (mastodon format) to our internal format.
    """
-   def fix_object(object) do
+   def fix_object(object, options \\ []) do
      object
      |> fix_actor
      |> fix_url
      |> fix_attachments
      |> fix_context
-     |> fix_in_reply_to
+     |> fix_in_reply_to(options)
      |> fix_emoji
      |> fix_tag
      |> fix_content_map
      |> fix_likes
      |> fix_addressing
      |> fix_summary
-     |> fix_type
+     |> fix_type(options)
    end
  
    def fix_summary(%{"summary" => nil} = object) do
      object
    end
  
-   def fix_in_reply_to(%{"inReplyTo" => in_reply_to} = object)
+   def fix_in_reply_to(object, options \\ [])
+   def fix_in_reply_to(%{"inReplyTo" => in_reply_to} = object, options)
        when not is_nil(in_reply_to) do
      in_reply_to_id =
        cond do
            ""
        end
  
-     case get_obj_helper(in_reply_to_id) do
-       {:ok, replied_object} ->
-         with %Activity{} = _activity <-
-                Activity.get_create_by_object_ap_id(replied_object.data["id"]) do
-           object
-           |> Map.put("inReplyTo", replied_object.data["id"])
-           |> Map.put("inReplyToAtomUri", object["inReplyToAtomUri"] || in_reply_to_id)
-           |> Map.put("conversation", replied_object.data["context"] || object["conversation"])
-           |> Map.put("context", replied_object.data["context"] || object["conversation"])
-         else
-           e ->
-             Logger.error("Couldn't fetch \"#{inspect(in_reply_to_id)}\", error: #{inspect(e)}")
-             object
-         end
+     object = Map.put(object, "inReplyToAtomUri", in_reply_to_id)
  
-       e ->
-         Logger.error("Couldn't fetch \"#{inspect(in_reply_to_id)}\", error: #{inspect(e)}")
-         object
+     if Federator.allowed_incoming_reply_depth?(options[:depth]) do
+       case get_obj_helper(in_reply_to_id, options) do
+         {:ok, replied_object} ->
+           with %Activity{} = _activity <-
+                  Activity.get_create_by_object_ap_id(replied_object.data["id"]) do
+             object
+             |> Map.put("inReplyTo", replied_object.data["id"])
+             |> Map.put("inReplyToAtomUri", object["inReplyToAtomUri"] || in_reply_to_id)
+             |> Map.put("conversation", replied_object.data["context"] || object["conversation"])
+             |> Map.put("context", replied_object.data["context"] || object["conversation"])
+           else
+             e ->
+               Logger.error("Couldn't fetch \"#{inspect(in_reply_to_id)}\", error: #{inspect(e)}")
+               object
+           end
+         e ->
+           Logger.error("Couldn't fetch \"#{inspect(in_reply_to_id)}\", error: #{inspect(e)}")
+           object
+       end
+     else
+       object
      end
    end
  
-   def fix_in_reply_to(object), do: object
+   def fix_in_reply_to(object, _options), do: object
  
    def fix_context(object) do
      context = object["context"] || object["conversation"] || Utils.generate_context_id()
  
    def fix_content_map(object), do: object
  
-   def fix_type(%{"inReplyTo" => reply_id} = object) when is_binary(reply_id) do
-     reply = Object.normalize(reply_id)
+   def fix_type(object, options \\ [])
+   def fix_type(%{"inReplyTo" => reply_id} = object, options) when is_binary(reply_id) do
+     reply =
+       if Federator.allowed_incoming_reply_depth?(options[:depth]) do
+         Object.normalize(reply_id, true)
+       end
  
-     if reply.data["type"] == "Question" and object["name"] do
+     if reply && (reply.data["type"] == "Question" and object["name"]) do
        Map.put(object, "type", "Answer")
      else
        object
      end
    end
  
-   def fix_type(object), do: object
+   def fix_type(object, _), do: object
  
    defp mastodon_follow_hack(%{"id" => id, "actor" => follower_id}, followed) do
      with true <- id =~ "follows",
      end
    end
  
+   def handle_incoming(data, options \\ [])
    # Flag objects are placed ahead of the ID check because Mastodon 2.8 and earlier send them
    # with nil ID.
-   def handle_incoming(%{"type" => "Flag", "object" => objects, "actor" => actor} = data) do
+   def handle_incoming(%{"type" => "Flag", "object" => objects, "actor" => actor} = data, _options) do
      with context <- data["context"] || Utils.generate_context_id(),
           content <- data["content"] || "",
           %User{} = actor <- User.get_cached_by_ap_id(actor),
    end
  
    # disallow objects with bogus IDs
-   def handle_incoming(%{"id" => nil}), do: :error
-   def handle_incoming(%{"id" => ""}), do: :error
+   def handle_incoming(%{"id" => nil}, _options), do: :error
+   def handle_incoming(%{"id" => ""}, _options), do: :error
    # length of https:// = 8, should validate better, but good enough for now.
-   def handle_incoming(%{"id" => id}) when not (is_binary(id) and length(id) > 8), do: :error
+   def handle_incoming(%{"id" => id}, _options) when not (is_binary(id) and length(id) > 8),
+     do: :error
  
    # TODO: validate those with a Ecto scheme
    # - tags
    # - emoji
-   def handle_incoming(%{"type" => "Create", "object" => %{"type" => objtype} = object} = data)
+   def handle_incoming(
+         %{"type" => "Create", "object" => %{"type" => objtype} = object} = data,
+         options
+       )
        when objtype in ["Article", "Note", "Video", "Page", "Question", "Answer"] do
      actor = Containment.get_actor(data)
  
  
      with nil <- Activity.get_create_by_object_ap_id(object["id"]),
           {:ok, %User{} = user} <- User.get_or_fetch_by_ap_id(data["actor"]) do
-       object = fix_object(data["object"])
+       options = Keyword.put(options, :depth, (options[:depth] || 0) + 1)
+       object = fix_object(data["object"], options)
  
        params = %{
          to: data["to"],
    end
  
    def handle_incoming(
-         %{"type" => "Follow", "object" => followed, "actor" => follower, "id" => id} = data
+         %{"type" => "Follow", "object" => followed, "actor" => follower, "id" => id} = data,
+         _options
        ) do
      with %User{local: true} = followed <- User.get_cached_by_ap_id(followed),
           {:ok, %User{} = follower} <- User.get_or_fetch_by_ap_id(follower),
           {:ok, activity} <- ActivityPub.follow(follower, followed, id, false) do
        with deny_follow_blocked <- Pleroma.Config.get([:user, :deny_follow_blocked]),
-            {:user_blocked, false} <-
+            {_, false} <-
               {:user_blocked, User.blocks?(followed, follower) && deny_follow_blocked},
-            {:user_locked, false} <- {:user_locked, User.locked?(followed)},
-            {:follow, {:ok, follower}} <- {:follow, User.follow(follower, followed)} do
+            {_, false} <- {:user_locked, User.locked?(followed)},
+            {_, {:ok, follower}} <- {:follow, User.follow(follower, followed)},
+            {_, {:ok, _}} <-
+              {:follow_state_update, Utils.update_follow_state_for_all(activity, "accept")} do
          ActivityPub.accept(%{
            to: [follower.ap_id],
            actor: followed,
          })
        else
          {:user_blocked, true} ->
-           {:ok, _} = Utils.update_follow_state(activity, "reject")
+           {:ok, _} = Utils.update_follow_state_for_all(activity, "reject")
  
            ActivityPub.reject(%{
              to: [follower.ap_id],
            })
  
          {:follow, {:error, _}} ->
-           {:ok, _} = Utils.update_follow_state(activity, "reject")
+           {:ok, _} = Utils.update_follow_state_for_all(activity, "reject")
  
            ActivityPub.reject(%{
              to: [follower.ap_id],
    end
  
    def handle_incoming(
-         %{"type" => "Accept", "object" => follow_object, "actor" => _actor, "id" => _id} = data
+         %{"type" => "Accept", "object" => follow_object, "actor" => _actor, "id" => _id} = data,
+         _options
        ) do
      with actor <- Containment.get_actor(data),
           {:ok, %User{} = followed} <- User.get_or_fetch_by_ap_id(actor),
           {:ok, follow_activity} <- get_follow_activity(follow_object, followed),
-          {:ok, follow_activity} <- Utils.update_follow_state(follow_activity, "accept"),
+          {:ok, follow_activity} <- Utils.update_follow_state_for_all(follow_activity, "accept"),
           %User{local: true} = follower <- User.get_cached_by_ap_id(follow_activity.data["actor"]),
-          {:ok, activity} <-
-            ActivityPub.accept(%{
-              to: follow_activity.data["to"],
-              type: "Accept",
-              actor: followed,
-              object: follow_activity.data["id"],
-              local: false
-            }) do
-       if not User.following?(follower, followed) do
-         {:ok, _follower} = User.follow(follower, followed)
-       end
-       {:ok, activity}
+          {:ok, _follower} = User.follow(follower, followed) do
+       ActivityPub.accept(%{
+         to: follow_activity.data["to"],
+         type: "Accept",
+         actor: followed,
+         object: follow_activity.data["id"],
+         local: false
+       })
      else
        _e -> :error
      end
    end
  
    def handle_incoming(
-         %{"type" => "Reject", "object" => follow_object, "actor" => _actor, "id" => _id} = data
+         %{"type" => "Reject", "object" => follow_object, "actor" => _actor, "id" => _id} = data,
+         _options
        ) do
      with actor <- Containment.get_actor(data),
           {:ok, %User{} = followed} <- User.get_or_fetch_by_ap_id(actor),
           {:ok, follow_activity} <- get_follow_activity(follow_object, followed),
-          {:ok, follow_activity} <- Utils.update_follow_state(follow_activity, "reject"),
+          {:ok, follow_activity} <- Utils.update_follow_state_for_all(follow_activity, "reject"),
           %User{local: true} = follower <- User.get_cached_by_ap_id(follow_activity.data["actor"]),
           {:ok, activity} <-
             ActivityPub.reject(%{
    end
  
    def handle_incoming(
-         %{"type" => "Like", "object" => object_id, "actor" => _actor, "id" => id} = data
+         %{"type" => "Like", "object" => object_id, "actor" => _actor, "id" => id} = data,
+         _options
        ) do
      with actor <- Containment.get_actor(data),
           {:ok, %User{} = actor} <- User.get_or_fetch_by_ap_id(actor),
    end
  
    def handle_incoming(
-         %{"type" => "Announce", "object" => object_id, "actor" => _actor, "id" => id} = data
+         %{"type" => "Announce", "object" => object_id, "actor" => _actor, "id" => id} = data,
+         _options
        ) do
      with actor <- Containment.get_actor(data),
           {:ok, %User{} = actor} <- User.get_or_fetch_by_ap_id(actor),
  
    def handle_incoming(
          %{"type" => "Update", "object" => %{"type" => object_type} = object, "actor" => actor_id} =
-           data
+           data,
+         _options
        )
        when object_type in ["Person", "Application", "Service", "Organization"] do
      with %User{ap_id: ^actor_id} = actor <- User.get_cached_by_ap_id(object["id"]) do
    # an error or a tombstone.  This would allow us to verify that a deletion actually took
    # place.
    def handle_incoming(
-         %{"type" => "Delete", "object" => object_id, "actor" => _actor, "id" => _id} = data
+         %{"type" => "Delete", "object" => object_id, "actor" => actor, "id" => _id} = data,
+         _options
        ) do
      object_id = Utils.get_ap_id(object_id)
  
           {:ok, activity} <- ActivityPub.delete(object, false) do
        {:ok, activity}
      else
-       _e -> :error
+       nil ->
+         case User.get_cached_by_ap_id(object_id) do
+           %User{ap_id: ^actor} = user ->
+             {:ok, followers} = User.get_followers(user)
+             Enum.each(followers, fn follower ->
+               User.unfollow(follower, user)
+             end)
+             {:ok, friends} = User.get_friends(user)
+             Enum.each(friends, fn followed ->
+               User.unfollow(user, followed)
+             end)
+             User.invalidate_cache(user)
+             Repo.delete(user)
+           nil ->
+             :error
+         end
+       _e ->
+         :error
      end
    end
  
            "object" => %{"type" => "Announce", "object" => object_id},
            "actor" => _actor,
            "id" => id
-         } = data
+         } = data,
+         _options
        ) do
      with actor <- Containment.get_actor(data),
           {:ok, %User{} = actor} <- User.get_or_fetch_by_ap_id(actor),
            "object" => %{"type" => "Follow", "object" => followed},
            "actor" => follower,
            "id" => id
-         } = _data
+         } = _data,
+         _options
        ) do
      with %User{local: true} = followed <- User.get_cached_by_ap_id(followed),
           {:ok, %User{} = follower} <- User.get_or_fetch_by_ap_id(follower),
            "object" => %{"type" => "Block", "object" => blocked},
            "actor" => blocker,
            "id" => id
-         } = _data
+         } = _data,
+         _options
        ) do
      with true <- Pleroma.Config.get([:activitypub, :accept_blocks]),
           %User{local: true} = blocked <- User.get_cached_by_ap_id(blocked),
    end
  
    def handle_incoming(
-         %{"type" => "Block", "object" => blocked, "actor" => blocker, "id" => id} = _data
+         %{"type" => "Block", "object" => blocked, "actor" => blocker, "id" => id} = _data,
+         _options
        ) do
      with true <- Pleroma.Config.get([:activitypub, :accept_blocks]),
           %User{local: true} = blocked = User.get_cached_by_ap_id(blocked),
            "object" => %{"type" => "Like", "object" => object_id},
            "actor" => _actor,
            "id" => id
-         } = data
+         } = data,
+         _options
        ) do
      with actor <- Containment.get_actor(data),
           {:ok, %User{} = actor} <- User.get_or_fetch_by_ap_id(actor),
      end
    end
  
-   def handle_incoming(_), do: :error
+   def handle_incoming(_, _), do: :error
  
-   def get_obj_helper(id) do
-     if object = Object.normalize(id), do: {:ok, object}, else: nil
+   def get_obj_helper(id, options \\ []) do
+     if object = Object.normalize(id, true, options), do: {:ok, object}, else: nil
    end
  
    def set_reply_to_uri(%{"inReplyTo" => in_reply_to} = object) when is_binary(in_reply_to) do
  
    def prepare_outgoing(%{"type" => "Create", "object" => object_id} = data) do
      object =
 -      Object.normalize(object_id).data
 +      object_id
 +      |> Object.normalize()
 +      |> Map.get(:data)
        |> prepare_object
  
      data =
        data
        |> Map.put("object", object)
        |> Map.merge(Utils.make_json_ld_header())
 +      |> Map.delete("bcc")
  
      {:ok, data}
    end
index 25b5fedb893b09d9a33559848af497144e0857ff,f1450b1139178d940a78f854232613b67980ca9c..8e3892bdf778d3743c9219d34551b7b817d00f51
@@@ -11,7 -11,9 +11,9 @@@ defmodule Pleroma.Web.CommonAPI d
    alias Pleroma.User
    alias Pleroma.Web.ActivityPub.ActivityPub
    alias Pleroma.Web.ActivityPub.Utils
+   alias Pleroma.Web.ActivityPub.Visibility
  
+   import Pleroma.Web.Gettext
    import Pleroma.Web.CommonAPI.Utils
  
    def follow(follower, followed) do
@@@ -35,9 -37,9 +37,9 @@@
    end
  
    def accept_follow_request(follower, followed) do
-     with {:ok, follower} <- User.maybe_follow(follower, followed),
+     with {:ok, follower} <- User.follow(follower, followed),
           %Activity{} = follow_activity <- Utils.fetch_latest_follow(follower, followed),
-          {:ok, follow_activity} <- Utils.update_follow_state(follow_activity, "accept"),
+          {:ok, follow_activity} <- Utils.update_follow_state_for_all(follow_activity, "accept"),
           {:ok, _activity} <-
             ActivityPub.accept(%{
               to: [follower.ap_id],
@@@ -51,7 -53,7 +53,7 @@@
  
    def reject_follow_request(follower, followed) do
      with %Activity{} = follow_activity <- Utils.fetch_latest_follow(follower, followed),
-          {:ok, follow_activity} <- Utils.update_follow_state(follow_activity, "reject"),
+          {:ok, follow_activity} <- Utils.update_follow_state_for_all(follow_activity, "reject"),
           {:ok, _activity} <-
             ActivityPub.reject(%{
               to: [follower.ap_id],
@@@ -73,7 -75,7 +75,7 @@@
        {:ok, delete}
      else
        _ ->
-         {:error, "Could not delete"}
+         {:error, dgettext("errors", "Could not delete")}
      end
    end
  
@@@ -84,7 -86,7 +86,7 @@@
        ActivityPub.announce(user, object)
      else
        _ ->
-         {:error, "Could not repeat"}
+         {:error, dgettext("errors", "Could not repeat")}
      end
    end
  
@@@ -94,7 -96,7 +96,7 @@@
        ActivityPub.unannounce(user, object)
      else
        _ ->
-         {:error, "Could not unrepeat"}
+         {:error, dgettext("errors", "Could not unrepeat")}
      end
    end
  
        ActivityPub.like(user, object)
      else
        _ ->
-         {:error, "Could not favorite"}
+         {:error, dgettext("errors", "Could not favorite")}
      end
    end
  
        ActivityPub.unlike(user, object)
      else
        _ ->
-         {:error, "Could not unfavorite"}
+         {:error, dgettext("errors", "Could not unfavorite")}
      end
    end
  
        object = Object.get_cached_by_ap_id(object.data["id"])
        {:ok, answer_activities, object}
      else
-       {:author, _} -> {:error, "Poll's author can't vote"}
-       {:existing_votes, _} -> {:error, "Already voted"}
-       {:choice_check, {_, false}} -> {:error, "Invalid indices"}
-       {:count_check, false} -> {:error, "Too many choices"}
+       {:author, _} -> {:error, dgettext("errors", "Poll's author can't vote")}
+       {:existing_votes, _} -> {:error, dgettext("errors", "Already voted")}
+       {:choice_check, {_, false}} -> {:error, dgettext("errors", "Invalid indices")}
+       {:count_check, false} -> {:error, dgettext("errors", "Too many choices")}
      end
    end
  
        when visibility in ~w{public unlisted private direct},
        do: {visibility, get_replied_to_visibility(in_reply_to)}
  
 +  def get_visibility(%{"visibility" => "list:" <> list_id}, in_reply_to) do
 +    visibility = {:list, String.to_integer(list_id)}
 +    {visibility, get_replied_to_visibility(in_reply_to)}
 +  end
 +
    def get_visibility(_, in_reply_to) when not is_nil(in_reply_to) do
      visibility = get_replied_to_visibility(in_reply_to)
      {visibility, visibility}
           addressed_users <- get_addressed_users(mentioned_users, data["to"]),
           {poll, poll_emoji} <- make_poll_data(data),
           {to, cc} <- get_to_and_cc(user, addressed_users, in_reply_to, visibility),
 +         bcc <- bcc_for_list(user, visibility),
           context <- make_context(in_reply_to),
           cw <- data["spoiler_text"] || "",
           sensitive <- data["sensitive"] || Enum.member?(tags, {"#nsfw", "nsfw"}),
           full_payload <- String.trim(status <> cw),
-          length when length in 1..limit <- String.length(full_payload),
+          :ok <- validate_character_limit(full_payload, attachments, limit),
           object <-
             make_note_data(
               user.ap_id,
               "emoji",
               Map.merge(Formatter.get_emoji_map(full_payload), poll_emoji)
             ) do
 -      res =
 -        ActivityPub.create(
 -          %{
 -            to: to,
 -            actor: user,
 -            context: context,
 -            object: object,
 -            additional: %{"cc" => cc, "directMessage" => visibility == "direct"}
 -          },
 -          Pleroma.Web.ControllerHelper.truthy_param?(data["preview"]) || false
 -        )
 -
 -      res
 +      ActivityPub.create(
 +        %{
 +          to: to,
 +          actor: user,
 +          context: context,
 +          object: object,
 +          additional: %{"cc" => cc, "bcc" => bcc, "directMessage" => visibility == "direct"}
 +        },
 +        Pleroma.Web.ControllerHelper.truthy_param?(data["preview"]) || false
 +      )
      else
-       e -> {:error, e}
+       {:private_to_public, true} ->
+         {:error, dgettext("errors", "The message visibility must be direct")}
+       {:error, _} = e ->
+         e
+       e ->
+         {:error, e}
      end
    end
  
             },
             object: %Object{
               data: %{
-                "to" => object_to,
                 "type" => "Note"
               }
             }
           } = activity <- get_by_id_or_ap_id(id_or_ap_id),
-          true <- Enum.member?(object_to, "https://www.w3.org/ns/activitystreams#Public"),
+          true <- Visibility.is_public?(activity),
           %{valid?: true} = info_changeset <-
             User.Info.add_pinnned_activity(user.info, activity),
           changeset <-
          {:error, err}
  
        _ ->
-         {:error, "Could not pin"}
+         {:error, dgettext("errors", "Could not pin")}
      end
    end
  
          {:error, err}
  
        _ ->
-         {:error, "Could not unpin"}
+         {:error, dgettext("errors", "Could not unpin")}
      end
    end
  
      with {:ok, _} <- ThreadMute.add_mute(user.id, activity.data["context"]) do
        {:ok, activity}
      else
-       {:error, _} -> {:error, "conversation is already muted"}
+       {:error, _} -> {:error, dgettext("errors", "conversation is already muted")}
      end
    end
  
        {:ok, activity}
      else
        {:error, err} -> {:error, err}
-       {:account_id, %{}} -> {:error, "Valid `account_id` required"}
-       {:account, nil} -> {:error, "Account not found"}
+       {:account_id, %{}} -> {:error, dgettext("errors", "Valid `account_id` required")}
+       {:account, nil} -> {:error, dgettext("errors", "Account not found")}
      end
    end
  
           {:ok, activity} <- Utils.update_report_state(activity, state) do
        {:ok, activity}
      else
-       nil ->
-         {:error, :not_found}
-       {:error, reason} ->
-         {:error, reason}
-       _ ->
-         {:error, "Could not update state"}
+       nil -> {:error, :not_found}
+       {:error, reason} -> {:error, reason}
+       _ -> {:error, dgettext("errors", "Could not update state")}
      end
    end
  
           {:ok, activity} <- set_visibility(activity, opts) do
        {:ok, activity}
      else
-       nil ->
-         {:error, :not_found}
-       {:error, reason} ->
-         {:error, reason}
+       nil -> {:error, :not_found}
+       {:error, reason} -> {:error, reason}
      end
    end
  
index 6c9e117ae1c42cc722992ca39323733202d1eec3,8e482eef7b88f9e9520e40e70b039598e0ee3d01..d4bfdd7e40fd55cec353391f2295d01ed7ceab86
@@@ -3,6 -3,8 +3,8 @@@
  # SPDX-License-Identifier: AGPL-3.0-only
  
  defmodule Pleroma.Web.CommonAPI.Utils do
+   import Pleroma.Web.Gettext
    alias Calendar.Strftime
    alias Comeonin.Pbkdf2
    alias Pleroma.Activity
      end
    end
  
 +  def get_to_and_cc(_user, _mentions, _inReplyTo, _), do: {[], []}
 +
    def get_addressed_users(_, to) when is_list(to) do
      User.get_ap_ids_by_nicknames(to)
    end
  
    def get_addressed_users(mentioned_users, _), do: mentioned_users
  
 +  def bcc_for_list(user, {:list, list_id}) do
 +    list = Pleroma.List.get(list_id, user)
 +    [list.ap_id]
 +  end
 +
 +  def bcc_for_list(_, _), do: []
 +
    def make_poll_data(%{"poll" => %{"options" => options, "expires_in" => expires_in}} = data)
        when is_list(options) do
      %{max_expiration: max_expiration, min_expiration: min_expiration} =
           true <- Pbkdf2.checkpw(password, db_user.password_hash) do
        {:ok, db_user}
      else
-       _ -> {:error, "Invalid password."}
+       _ -> {:error, dgettext("errors", "Invalid password.")}
      end
    end
  
      if String.length(comment) <= max_size do
        {:ok, format_input(comment, "text/plain")}
      else
-       {:error, "Comment must be up to #{max_size} characters"}
+       {:error,
+        dgettext("errors", "Comment must be up to %{max_size} characters", max_size: max_size)}
      end
    end
  
        context
      else
        _e ->
-         {:error, "No such conversation"}
+         {:error, dgettext("errors", "No such conversation")}
      end
    end
  
        "inReplyTo" => object.data["id"]
      }
    end
+   def validate_character_limit(full_payload, attachments, limit) do
+     length = String.length(full_payload)
+     if length < limit do
+       if length > 0 or Enum.count(attachments) > 0 do
+         :ok
+       else
+         {:error, dgettext("errors", "Cannot post an empty status without attachments")}
+       end
+     else
+       {:error, dgettext("errors", "The status is over the character limit")}
+     end
+   end
  end
index 19e3ef401088d8dda75ebadb545d3364f523ea4d,e96e4e1e411e489ace59316c5fcebdb8d1e1ab4f..9b01ebcc642ea30acc7f09024bc5f7c9c0fea58b
@@@ -123,26 -123,11 +123,26 @@@ defmodule Pleroma.Web.Salmon d
      {:ok, salmon}
    end
  
 -  def remote_users(%{data: %{"to" => to} = data}) do
 -    to = to ++ (data["cc"] || [])
 -
 -    to
 -    |> Enum.map(fn id -> User.get_cached_by_ap_id(id) end)
 +  def remote_users(%User{id: user_id}, %{data: %{"to" => to} = data}) do
 +    cc = Map.get(data, "cc", [])
 +
 +    bcc =
 +      data
 +      |> Map.get("bcc", [])
 +      |> Enum.reduce([], fn ap_id, bcc ->
 +        case Pleroma.List.get_by_ap_id(ap_id) do
 +          %Pleroma.List{user_id: ^user_id} = list ->
 +            {:ok, following} = Pleroma.List.get_following(list)
 +            bcc ++ Enum.map(following, & &1.ap_id)
 +
 +          _ ->
 +            bcc
 +        end
 +      end)
 +
 +    [to, cc, bcc]
 +    |> Enum.concat()
 +    |> Enum.map(&User.get_cached_by_ap_id/1)
      |> Enum.filter(fn user -> user && !user.local end)
    end
  
          do: Instances.set_reachable(url)
  
        Logger.debug(fn -> "Pushed to #{url}, code #{code}" end)
-       :ok
+       {:ok, code}
      else
        e ->
          unless params[:unreachable_since], do: Instances.set_reachable(url)
        {:ok, private, _} = Keys.keys_from_pem(keys)
        {:ok, feed} = encode(private, feed)
  
 -      remote_users = remote_users(activity)
 +      remote_users = remote_users(user, activity)
  
        salmon_urls = Enum.map(remote_users, & &1.info.salmon)
        reachable_urls_metadata = Instances.filter_reachable(salmon_urls)
index 2bed1563937d3a3dd5f7d409bb212d2d84f6479e,59d56f3a758324633b0d70953ebd076f5117f3ca..00adbc0f9a7425d2b684b1fb7ed6a7fa755ae303
@@@ -254,10 -254,8 +254,8 @@@ defmodule Pleroma.Web.ActivityPub.Activ
        }
  
        {:ok, %Activity{} = activity} = ActivityPub.insert(data)
-       object = Object.normalize(activity.data["object"])
+       assert object = Object.normalize(activity)
        assert is_binary(object.data["id"])
-       assert %Object{} = Object.get_by_ap_id(activity.data["object"])
      end
    end
  
    describe "like an object" do
      test "adds a like activity to the db" do
        note_activity = insert(:note_activity)
-       object = Object.get_by_ap_id(note_activity.data["object"]["id"])
+       assert object = Object.normalize(note_activity)
        user = insert(:user)
        user_two = insert(:user)
  
  
        assert like_activity == same_like_activity
        assert object.data["likes"] == [user.ap_id]
+       assert object.data["like_count"] == 1
  
        [note_activity] = Activity.get_all_create_by_object_ap_id(object.data["id"])
        assert note_activity.data["object"]["like_count"] == 1
  
        {:ok, _like_activity, object} = ActivityPub.like(user_two, object)
        assert object.data["like_count"] == 2
+       [note_activity] = Activity.get_all_create_by_object_ap_id(object.data["id"])
+       assert note_activity.data["object"]["like_count"] == 2
      end
    end
  
    describe "unliking" do
      test "unliking a previously liked object" do
        note_activity = insert(:note_activity)
-       object = Object.get_by_ap_id(note_activity.data["object"]["id"])
+       object = Object.normalize(note_activity)
        user = insert(:user)
  
        # Unliking something that hasn't been liked does nothing
    describe "announcing an object" do
      test "adds an announce activity to the db" do
        note_activity = insert(:note_activity)
-       object = Object.get_by_ap_id(note_activity.data["object"]["id"])
+       object = Object.normalize(note_activity)
        user = insert(:user)
  
        {:ok, announce_activity, object} = ActivityPub.announce(user, object)
    describe "unannouncing an object" do
      test "unannouncing a previously announced object" do
        note_activity = insert(:note_activity)
-       object = Object.get_by_ap_id(note_activity.data["object"]["id"])
+       object = Object.normalize(note_activity)
        user = insert(:user)
  
        # Unannouncing an object that is not announced does nothing
        assert activity.data["type"] == "Undo"
        assert activity.data["actor"] == follower.ap_id
  
-       assert is_map(activity.data["object"])
-       assert activity.data["object"]["type"] == "Follow"
-       assert activity.data["object"]["object"] == followed.ap_id
-       assert activity.data["object"]["id"] == follow_activity.data["id"]
+       embedded_object = activity.data["object"]
+       assert is_map(embedded_object)
+       assert embedded_object["type"] == "Follow"
+       assert embedded_object["object"] == followed.ap_id
+       assert embedded_object["id"] == follow_activity.data["id"]
      end
    end
  
        assert activity.data["type"] == "Undo"
        assert activity.data["actor"] == blocker.ap_id
  
-       assert is_map(activity.data["object"])
-       assert activity.data["object"]["type"] == "Block"
-       assert activity.data["object"]["object"] == blocked.ap_id
-       assert activity.data["object"]["id"] == block_activity.data["id"]
+       embedded_object = activity.data["object"]
+       assert is_map(embedded_object)
+       assert embedded_object["type"] == "Block"
+       assert embedded_object["object"] == blocked.ap_id
+       assert embedded_object["id"] == block_activity.data["id"]
      end
    end
  
    describe "deletion" do
      test "it creates a delete activity and deletes the original object" do
        note = insert(:note_activity)
-       object = Object.get_by_ap_id(note.data["object"]["id"])
+       object = Object.normalize(note)
        {:ok, delete} = ActivityPub.delete(object)
  
        assert delete.data["type"] == "Delete"
        assert delete.data["actor"] == note.data["actor"]
-       assert delete.data["object"] == note.data["object"]["id"]
+       assert delete.data["object"] == object.data["id"]
  
        assert Activity.get_by_id(delete.id) != nil
  
      test "it creates a delete activity and checks that it is also sent to users mentioned by the deleted object" do
        user = insert(:user)
        note = insert(:note_activity)
+       object = Object.normalize(note)
  
        {:ok, object} =
-         Object.get_by_ap_id(note.data["object"]["id"])
+         object
          |> Object.change(%{
            data: %{
-             "actor" => note.data["object"]["actor"],
-             "id" => note.data["object"]["id"],
+             "actor" => object.data["actor"],
+             "id" => object.data["id"],
              "to" => [user.ap_id],
              "type" => "Note"
            }
  
        assert update.data["actor"] == user.ap_id
        assert update.data["to"] == [user.follower_address]
-       assert update.data["object"]["id"] == user_data["id"]
-       assert update.data["object"]["type"] == user_data["type"]
+       assert embedded_object = update.data["object"]
+       assert embedded_object["id"] == user_data["id"]
+       assert embedded_object["type"] == user_data["type"]
      end
    end
  
      end
    end
  
 +  test "fetch_activities/2 returns activities addressed to a list " do
 +    user = insert(:user)
 +    member = insert(:user)
 +    {:ok, list} = Pleroma.List.create("foo", user)
 +    {:ok, list} = Pleroma.List.follow(list, member)
 +
 +    {:ok, activity} =
 +      CommonAPI.post(user, %{"status" => "foobar", "visibility" => "list:#{list.id}"})
 +
 +    activity = Repo.preload(activity, :bookmark)
 +    activity = %Activity{activity | thread_muted?: !!activity.thread_muted?}
 +
 +    assert ActivityPub.fetch_activities([], %{"user" => user}) == [activity]
 +  end
 +
    def data_uri do
      File.read!("test/fixtures/avatar_data_uri")
    end
index e6388f88af2671e9f9fc77dde4b89bf5934299ec,825e998793cf6170ea99f37d5ea1c8806c3591b8..eb9300769d3547ba825fde5b52c3bc171160e588
@@@ -11,12 -11,13 +11,13 @@@ defmodule Pleroma.Web.ActivityPub.Trans
    alias Pleroma.User
    alias Pleroma.Web.ActivityPub.ActivityPub
    alias Pleroma.Web.ActivityPub.Transmogrifier
-   alias Pleroma.Web.ActivityPub.Utils
+   alias Pleroma.Web.CommonAPI
    alias Pleroma.Web.OStatus
    alias Pleroma.Web.Websub.WebsubClientSubscription
  
+   import Mock
    import Pleroma.Factory
-   alias Pleroma.Web.CommonAPI
+   import ExUnit.CaptureLog
  
    setup_all do
      Tesla.Mock.mock_global(fn env -> apply(HttpRequestMock, :request, [env]) end)
@@@ -30,7 -31,7 +31,7 @@@
        data =
          File.read!("test/fixtures/mastodon-post-activity.json")
          |> Poison.decode!()
-         |> Map.put("object", activity.data["object"])
+         |> Map.put("object", Object.normalize(activity).data)
  
        {:ok, returned_activity} = Transmogrifier.handle_incoming(data)
  
          data["object"]
          |> Map.put("inReplyTo", "https://shitposter.club/notice/2827873")
  
-       data =
-         data
-         |> Map.put("object", object)
+       data = Map.put(data, "object", object)
        {:ok, returned_activity} = Transmogrifier.handle_incoming(data)
-       returned_object = Object.normalize(returned_activity.data["object"])
+       returned_object = Object.normalize(returned_activity, false)
  
        assert activity =
                 Activity.get_create_by_object_ap_id(
        assert returned_object.data["inReplyToAtomUri"] == "https://shitposter.club/notice/2827873"
      end
  
+     test "it does not fetch replied-to activities beyond max_replies_depth" do
+       data =
+         File.read!("test/fixtures/mastodon-post-activity.json")
+         |> Poison.decode!()
+       object =
+         data["object"]
+         |> Map.put("inReplyTo", "https://shitposter.club/notice/2827873")
+       data = Map.put(data, "object", object)
+       with_mock Pleroma.Web.Federator,
+         allowed_incoming_reply_depth?: fn _ -> false end do
+         {:ok, returned_activity} = Transmogrifier.handle_incoming(data)
+         returned_object = Object.normalize(returned_activity, false)
+         refute Activity.get_create_by_object_ap_id(
+                  "tag:shitposter.club,2017-05-05:noticeId=2827873:objectType=comment"
+                )
+         assert returned_object.data["inReplyToAtomUri"] ==
+                  "https://shitposter.club/notice/2827873"
+       end
+     end
+     test "it does not crash if the object in inReplyTo can't be fetched" do
+       data =
+         File.read!("test/fixtures/mastodon-post-activity.json")
+         |> Poison.decode!()
+       object =
+         data["object"]
+         |> Map.put("inReplyTo", "https://404.site/whatever")
+       data =
+         data
+         |> Map.put("object", object)
+       assert capture_log(fn ->
+                {:ok, _returned_activity} = Transmogrifier.handle_incoming(data)
+              end) =~ "[error] Couldn't fetch \"\"https://404.site/whatever\"\", error: nil"
+     end
      test "it works for incoming notices" do
        data = File.read!("test/fixtures/mastodon-post-activity.json") |> Poison.decode!()
  
  
        assert data["actor"] == "http://mastodon.example.org/users/admin"
  
-       object = Object.normalize(data["object"]).data
-       assert object["id"] == "http://mastodon.example.org/users/admin/statuses/99512778738411822"
+       object_data = Object.normalize(data["object"]).data
+       assert object_data["id"] ==
+                "http://mastodon.example.org/users/admin/statuses/99512778738411822"
  
-       assert object["to"] == ["https://www.w3.org/ns/activitystreams#Public"]
+       assert object_data["to"] == ["https://www.w3.org/ns/activitystreams#Public"]
  
-       assert object["cc"] == [
+       assert object_data["cc"] == [
                 "http://mastodon.example.org/users/admin/followers",
                 "http://localtesting.pleroma.lol/users/lain"
               ]
  
-       assert object["actor"] == "http://mastodon.example.org/users/admin"
-       assert object["attributedTo"] == "http://mastodon.example.org/users/admin"
+       assert object_data["actor"] == "http://mastodon.example.org/users/admin"
+       assert object_data["attributedTo"] == "http://mastodon.example.org/users/admin"
  
-       assert object["context"] ==
+       assert object_data["context"] ==
                 "tag:mastodon.example.org,2018-02-12:objectId=20:objectType=Conversation"
  
-       assert object["sensitive"] == true
+       assert object_data["sensitive"] == true
  
-       user = User.get_cached_by_ap_id(object["actor"])
+       user = User.get_cached_by_ap_id(object_data["actor"])
  
        assert user.info.note_count == 1
      end
        assert object_data["cc"] == to
      end
  
-     test "it works for incoming follow requests" do
-       user = insert(:user)
-       data =
-         File.read!("test/fixtures/mastodon-follow-activity.json")
-         |> Poison.decode!()
-         |> Map.put("object", user.ap_id)
-       {:ok, %Activity{data: data, local: false}} = Transmogrifier.handle_incoming(data)
-       assert data["actor"] == "http://mastodon.example.org/users/admin"
-       assert data["type"] == "Follow"
-       assert data["id"] == "http://mastodon.example.org/users/admin#follows/2"
-       assert User.following?(User.get_cached_by_ap_id(data["actor"]), user)
-     end
-     test "it rejects incoming follow requests from blocked users when deny_follow_blocked is enabled" do
-       Pleroma.Config.put([:user, :deny_follow_blocked], true)
-       user = insert(:user)
-       {:ok, target} = User.get_or_fetch("http://mastodon.example.org/users/admin")
-       {:ok, user} = User.block(user, target)
-       data =
-         File.read!("test/fixtures/mastodon-follow-activity.json")
-         |> Poison.decode!()
-         |> Map.put("object", user.ap_id)
-       {:ok, %Activity{data: %{"id" => id}}} = Transmogrifier.handle_incoming(data)
-       %Activity{} = activity = Activity.get_by_ap_id(id)
-       assert activity.data["state"] == "reject"
-     end
-     test "it works for incoming follow requests from hubzilla" do
-       user = insert(:user)
-       data =
-         File.read!("test/fixtures/hubzilla-follow-activity.json")
-         |> Poison.decode!()
-         |> Map.put("object", user.ap_id)
-         |> Utils.normalize_params()
-       {:ok, %Activity{data: data, local: false}} = Transmogrifier.handle_incoming(data)
-       assert data["actor"] == "https://hubzilla.example.org/channel/kaniini"
-       assert data["type"] == "Follow"
-       assert data["id"] == "https://hubzilla.example.org/channel/kaniini#follows/2"
-       assert User.following?(User.get_cached_by_ap_id(data["actor"]), user)
-     end
      test "it works for incoming likes" do
        user = insert(:user)
        {:ok, activity} = CommonAPI.post(user, %{"status" => "hello"})
          data
          |> Map.put("object", object)
  
-       :error = Transmogrifier.handle_incoming(data)
+       assert capture_log(fn ->
+                :error = Transmogrifier.handle_incoming(data)
+              end) =~
+                "[error] Could not decode user at fetch http://mastodon.example.org/users/gargron, {:error, {:error, :nxdomain}}"
  
        assert Activity.get_by_id(activity.id)
      end
  
+     test "it works for incoming user deletes" do
+       %{ap_id: ap_id} = insert(:user, ap_id: "http://mastodon.example.org/users/admin")
+       data =
+         File.read!("test/fixtures/mastodon-delete-user.json")
+         |> Poison.decode!()
+       {:ok, _} = Transmogrifier.handle_incoming(data)
+       refute User.get_cached_by_ap_id(ap_id)
+     end
+     test "it fails for incoming user deletes with spoofed origin" do
+       %{ap_id: ap_id} = insert(:user)
+       data =
+         File.read!("test/fixtures/mastodon-delete-user.json")
+         |> Poison.decode!()
+         |> Map.put("actor", ap_id)
+       assert :error == Transmogrifier.handle_incoming(data)
+       assert User.get_cached_by_ap_id(ap_id)
+     end
      test "it works for incoming unannounces with an existing notice" do
        user = insert(:user)
        {:ok, activity} = CommonAPI.post(user, %{"status" => "hey"})
        {:ok, %Activity{data: data, local: false}} = Transmogrifier.handle_incoming(data)
  
        assert data["type"] == "Undo"
-       assert data["object"]["type"] == "Announce"
-       assert data["object"]["object"] == activity.data["object"]
+       assert object_data = data["object"]
+       assert object_data["type"] == "Announce"
+       assert object_data["object"] == activity.data["object"]
  
-       assert data["object"]["id"] ==
+       assert object_data["id"] ==
                 "http://mastodon.example.org/users/admin/statuses/99542391527669785/activity"
      end
  
        other_user = insert(:user)
  
        {:ok, activity} = CommonAPI.post(user, %{"status" => "test post"})
-       object = Object.normalize(activity.data["object"])
+       object = Object.normalize(activity)
  
        message = %{
          "@context" => "https://www.w3.org/ns/activitystreams",
  
        assert modified["directMessage"] == true
      end
 +
 +    test "it strips BCC field" do
 +      user = insert(:user)
 +      {:ok, list} = Pleroma.List.create("foo", user)
 +
 +      {:ok, activity} =
 +        CommonAPI.post(user, %{"status" => "foobar", "visibility" => "list:#{list.id}"})
 +
 +      {:ok, modified} = Transmogrifier.prepare_outgoing(activity.data)
 +
 +      assert is_nil(modified["bcc"])
 +    end
    end
  
    describe "user upgrade" do
index 9ef79e9c9d3f49ad9cfef5bdc511977c962e1510,958c931c49a21353d6cdf5f9ac8a286ec1f3c1e0..694b52356d621b45f1161d2e30c9a0a9ccd80732
@@@ -7,6 -7,7 +7,7 @@@ defmodule Pleroma.Web.CommonAPITest d
    alias Pleroma.Activity
    alias Pleroma.Object
    alias Pleroma.User
+   alias Pleroma.Web.ActivityPub.ActivityPub
    alias Pleroma.Web.CommonAPI
  
    import Pleroma.Factory
@@@ -33,7 -34,7 +34,7 @@@
      user = insert(:user)
      {:ok, activity} = CommonAPI.post(user, %{"status" => "#2hu #2HU"})
  
-     object = Object.normalize(activity.data["object"])
+     object = Object.normalize(activity)
  
      assert object.data["tag"] == ["2hu"]
    end
@@@ -86,7 -87,7 +87,7 @@@
            "content_type" => "text/html"
          })
  
-       object = Object.normalize(activity.data["object"])
+       object = Object.normalize(activity)
  
        assert object.data["content"] == "<p><b>2hu</b></p>alert('xss')"
      end
            "content_type" => "text/markdown"
          })
  
-       object = Object.normalize(activity.data["object"])
+       object = Object.normalize(activity)
  
        assert object.data["content"] == "<p><b>2hu</b></p>alert('xss')"
      end
                 })
  
        Enum.each(["public", "private", "unlisted"], fn visibility ->
-         assert {:error, {:private_to_public, _}} =
+         assert {:error, "The message visibility must be direct"} =
                   CommonAPI.post(user, %{
                     "status" => "suya..",
                     "visibility" => visibility,
                   })
        end)
      end
 +
 +    test "it allows to address a list" do
 +      user = insert(:user)
 +      {:ok, list} = Pleroma.List.create("foo", user)
 +
 +      {:ok, activity} =
 +        CommonAPI.post(user, %{"status" => "foobar", "visibility" => "list:#{list.id}"})
 +
 +      assert activity.data["bcc"] == [list.ap_id]
 +      assert activity.recipients == [list.ap_id, user.ap_id]
 +    end
    end
  
    describe "reactions" do
        assert %User{info: %{pinned_activities: [^id]}} = user
      end
  
+     test "unlisted statuses can be pinned", %{user: user} do
+       {:ok, activity} = CommonAPI.post(user, %{"status" => "HI!!!", "visibility" => "unlisted"})
+       assert {:ok, ^activity} = CommonAPI.pin(activity.id, user)
+     end
      test "only self-authored can be pinned", %{activity: activity} do
        user = insert(:user)
  
        assert User.showing_reblogs?(muter, muted) == true
      end
    end
+   describe "accept_follow_request/2" do
+     test "after acceptance, it sets all existing pending follow request states to 'accept'" do
+       user = insert(:user, info: %{locked: true})
+       follower = insert(:user)
+       follower_two = insert(:user)
+       {:ok, follow_activity} = ActivityPub.follow(follower, user)
+       {:ok, follow_activity_two} = ActivityPub.follow(follower, user)
+       {:ok, follow_activity_three} = ActivityPub.follow(follower_two, user)
+       assert follow_activity.data["state"] == "pending"
+       assert follow_activity_two.data["state"] == "pending"
+       assert follow_activity_three.data["state"] == "pending"
+       {:ok, _follower} = CommonAPI.accept_follow_request(follower, user)
+       assert Repo.get(Activity, follow_activity.id).data["state"] == "accept"
+       assert Repo.get(Activity, follow_activity_two.id).data["state"] == "accept"
+       assert Repo.get(Activity, follow_activity_three.id).data["state"] == "pending"
+     end
+     test "after rejection, it sets all existing pending follow request states to 'reject'" do
+       user = insert(:user, info: %{locked: true})
+       follower = insert(:user)
+       follower_two = insert(:user)
+       {:ok, follow_activity} = ActivityPub.follow(follower, user)
+       {:ok, follow_activity_two} = ActivityPub.follow(follower, user)
+       {:ok, follow_activity_three} = ActivityPub.follow(follower_two, user)
+       assert follow_activity.data["state"] == "pending"
+       assert follow_activity_two.data["state"] == "pending"
+       assert follow_activity_three.data["state"] == "pending"
+       {:ok, _follower} = CommonAPI.reject_follow_request(follower, user)
+       assert Repo.get(Activity, follow_activity.id).data["state"] == "reject"
+       assert Repo.get(Activity, follow_activity_two.id).data["state"] == "reject"
+       assert Repo.get(Activity, follow_activity_three.id).data["state"] == "pending"
+     end
+   end
  end