[feature] Allow "charset=utf8" in incoming AP POST requests (#2564)

* [feature] Allow "charset=utf8" in incoming AP POST requests

* changed my mind

* document POSTing to a GtS inbox

* correct link

docs/federation/federating_with_gotosocial.md

@@ -92,6 +92,32 @@ This ensures that remote servers cannot flood a GoToSocial instance with spuriou
 
 For more details on request throttling and rate limiting behavior, please see the [throttling](../api/throttling.md) and [rate limiting](../api/ratelimiting.md) documents.
 
+## Inbox
+
+GoToSocial implements Inboxes for Actors following the ActivityPub specification [here](https://www.w3.org/TR/activitypub/#inbox).
+
+Remote servers should deliver Activities to a GoToSocial server by making an HTTP POST request to each Inbox of the desired audience of an Activity, as described [here](https://www.w3.org/TR/activitypub/#delivery).
+
+GoToSocial accounts do not currently implement a [sharedInbox](https://www.w3.org/TR/activitypub/#shared-inbox-delivery) endpoint, though this is subject to change. Deduplication of delivered Activities, in case more than one Actor on a GoToSocial server is in the audience for an Activity, is handled on GoToSocial's side.
+
+POSTs to a GoToSocial Actor's inbox must be appropriately [http-signed](#http-signatures) by the delivering Actor.
+
+Accepted Inbox POST `Content-Type` headers are:
+
+- `application/activity+json`
+- `application/activity+json; charset=utf-8`
+- `application/ld+json; profile="https://www.w3.org/ns/activitystreams"`
+
+Inbox POST requests that do not use one of the above `Content-Type` headers will be rejected with HTTP status code [406 - Not Acceptable](https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/406).
+
+For more information on acceptable content types, see [the server-to-server interactions](https://www.w3.org/TR/activitypub/#server-to-server-interactions) section of the ActivityPub protocol.
+
+GoToSocial will return HTTP status code [202 - Accepted](https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/202) in response to validly-formed and signed Inbox POST requests.
+
+Invalidly-formed Inbox POST requests will receive a [400 - Bad Request](https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/400) HTTP status code in response. The response body may contain more information on why the GoToSocial server considered the request content to be badly formed. Other servers should not retry delivery of the Activity in case of a code `400` response.
+
+Even if GoToSocial returns a `202` status code, it may not continue processing the Activity delivered, depending on the originator(s), target(s) and type of the Activity. ActivityPub is an extensive protocol, and GoToSocial does not cover every combination of Activity and Object.
+
 ## Outbox
 
 GoToSocial implements Outboxes for Actors (ie., instance accounts) following the ActivityPub specification [here](https://www.w3.org/TR/activitypub/#outbox).

internal/federation/federatingactor.go

@@ -40,7 +40,7 @@
 // - application/activity+json
 // - application/ld+json;profile=https://w3.org/ns/activitystreams
 //
-// Where for the above we are leniant with whitespace and quotes.
+// Where for the above we are leniant with whitespace, quotes, and charset.
 func IsASMediaType(ct string) bool {
 	var (
 		// First content-type part,
@@ -48,7 +48,8 @@ func IsASMediaType(ct string) bool {
 		p1 string = ct //nolint:revive
 
 		// Second content-type part,
-		// contains AS IRI if provided
+		// contains AS IRI or charset
+		// if provided.
 		p2 string
 	)
 
@@ -56,7 +57,11 @@ func IsASMediaType(ct string) bool {
 	sep := strings.IndexByte(ct, ';')
 	if sep >= 0 {
 		p1 = ct[:sep]
+
+		// Trim all start/end
+		// space of second part.
 		p2 = ct[sep+1:]
+		p2 = strings.Trim(p2, " ")
 	}
 
 	// Trim any ending space from the
@@ -65,12 +70,12 @@ func IsASMediaType(ct string) bool {
 
 	switch p1 {
 	case "application/activity+json":
-		return p2 == ""
+		// Accept with or without charset.
+		// This should be case insensitive.
+		// https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Type#charset
+		return p2 == "" || strings.EqualFold(p2, "charset=utf-8")
 
 	case "application/ld+json":
-		// Trim all start/end space.
-		p2 = strings.Trim(p2, " ")
-
 		// Drop any quotes around the URI str.
 		p2 = strings.ReplaceAll(p2, "\"", "")
 

internal/federation/federatingactor_test.go

@@ -164,6 +164,22 @@ func TestIsASMediaType(t *testing.T) {
 			Input:  "application/activity+json",
 			Expect: true,
 		},
+		{
+			Input:  "application/activity+json; charset=utf-8",
+			Expect: true,
+		},
+		{
+			Input:  "application/activity+json;charset=utf-8",
+			Expect: true,
+		},
+		{
+			Input:  "application/activity+json ;charset=utf-8",
+			Expect: true,
+		},
+		{
+			Input:  "application/activity+json ; charset=utf-8",
+			Expect: true,
+		},
 		{
 			Input:  "application/ld+json;profile=https://www.w3.org/ns/activitystreams",
 			Expect: true,
@@ -196,6 +212,10 @@ func TestIsASMediaType(t *testing.T) {
 			Input:  "application/ld+json; profile=\"https://www.w3.org/ns/activitystreams\"",
 			Expect: true,
 		},
+		{
+			Input:  "application/ld+json",
+			Expect: false,
+		},
 	} {
 		if federation.IsASMediaType(test.Input) != test.Expect {
 			t.Errorf("did not get expected result %v for input: %s", test.Expect, test.Input)