sanitize html for statuses + instance (#97)

* sanitize html for statuses + instance

* sanitization

README.md

@@ -26,9 +26,7 @@ Tusky                                                        |  Pinafore
 :-----------------------------------------------------------:|:------------------------------------------------------------------:
 ![An image of GoToSocial in Tusky](./docs/assets/tusky.png)  | ![An image of GoToSocial in Pinafore](./docs/assets/pinafore.png)
 
-### Customizable
+### Granular post settings
-
-#### Granular post settings
 
 You should be able to choose how your posts can be interacted with:
 
@@ -37,40 +35,36 @@ You should be able to choose how your posts can be interacted with:
 * 'Likeable' toggle.
 * 'Replyable' toggle.
 
-#### Easy customizability for admins
+### Easy customizability for admins
 
 * Adjustable post length.
 * Media upload size settings.
 
-### Convenient
+### LetsEncrypt
-
-#### LetsEncrypt
 
  Built-in, automatic support for secure HTTPS with [LetsEncrypt](https://letsencrypt.org/).
 
-#### Light footprint and good performance
+### Light footprint and good performance
 
 Plays nice with lower-powered machines like Raspberry Pi, old laptops and tiny VPSes.
 
-#### Easy to deploy
+### Easy to deploy
 
 No external dependencies apart from a database. Just download the binary + assets (or Docker container), and run.
 
-### Secure
+### HTTP signature authentication
-
-#### HTTP signature authentication
 
 Protect your data.
 
-#### User Safety
+### User Safety
 
 Strict privacy enforcement for posts and strict blocking logic.
 
-#### Subscribeable and shareable allow/denylists for federation
+### Subscribeable and shareable allow/denylists for federation
 
 Import and export allowlists and denylists. Subscribe to community-created blocklists (think Adblocker, but for federation!).
 
-#### Various federation modes
+### Various federation modes
 
 * 'Normal' federation; discover new servers.
 * Allowlist-only federation; choose which servers you talk to.
@@ -82,6 +76,7 @@ These cool things will be implemented if time allows (because we really want the
 
 * **Groups** and group posting!
 * Reputation-based 'slow' federation.
+* Community decision making for federation and moderation actions.
 * User-selectable custom templates for rendering public posts:
   * Twitter-style
   * Blogpost
@@ -131,6 +126,7 @@ The following libraries and frameworks are used by GoToSocial, with gratitude
 * [google/uuid](https://github.com/google/uuid); UUID generation. [BSD-3-Clause License](https://spdx.org/licenses/BSD-3-Clause.html)
 * [gorilla/websocket](https://github.com/gorilla/websocket); Websocket connectivity. [BSD-2-Clause License](https://spdx.org/licenses/BSD-2-Clause.html).
 * [h2non/filetype](https://github.com/h2non/filetype); filetype checking. [MIT License](https://spdx.org/licenses/MIT.html).
+* [microcosm-cc/bluemonday](https://github.com/microcosm-cc/bluemonday); HTML user-input sanitization. [BSD-3-Clause License](https://spdx.org/licenses/BSD-3-Clause.html).
 * [oklog/ulid](https://github.com/oklog/ulid); sequential, database-friendly ID generation. [Apache-2.0 License](https://spdx.org/licenses/Apache-2.0.html).
 * [sirupsen/logrus](https://github.com/sirupsen/logrus); logging. [MIT License](https://spdx.org/licenses/MIT.html).
 * [stretchr/testify](https://github.com/stretchr/testify); test framework. [MIT License](https://spdx.org/licenses/MIT.html).
@@ -139,17 +135,17 @@ The following libraries and frameworks are used by GoToSocial, with gratitude
 * [urfave/cli](https://github.com/urfave/cli); command-line interface framework. [MIT License](https://spdx.org/licenses/MIT.html).
 * [wagslane/go-password-validator](https://github.com/wagslane/go-password-validator); password strength validation. [MIT License](https://spdx.org/licenses/MIT.html).
 
+### Image Attribution
+
+Sloth logo made by [Freepik](https://www.freepik.com) from [www.flaticon.com](https://www.flaticon.com/).
+
 ## Sponsorship + Funding
 
 Currently, this project is funded using Liberapay, to put bread on the table while work continues on it.
 
 If you want to sponsor this project, you can do so [here](https://liberapay.com/dumpsterqueer/)! `<3`
 
-### Image Attribution
+## License
-
-Sloth logo made by [Freepik](https://www.freepik.com) from [www.flaticon.com](https://www.flaticon.com/).
-
-### License
 
 GoToSocial is licensed under the [GNU AGPL v3 LICENSE](LICENSE).
 

go.mod

@@ -32,6 +32,7 @@ require (
 	github.com/json-iterator/go v1.1.11 // indirect
 	github.com/leodido/go-urn v1.2.1 // indirect
 	github.com/mattn/go-isatty v0.0.13 // indirect
+	github.com/microcosm-cc/bluemonday v1.0.15
 	github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
 	github.com/modern-go/reflect2 v1.0.1 // indirect
 	github.com/nfnt/resize v0.0.0-20180221191011-83c6a9932646
@@ -50,7 +51,6 @@ require (
 	github.com/vmihailenco/msgpack/v5 v5.3.4 // indirect
 	github.com/wagslane/go-password-validator v0.3.0
 	golang.org/x/crypto v0.0.0-20210513164829-c07d793c2f9a
-	golang.org/x/net v0.0.0-20210525063256-abc453219eb5 // indirect
 	golang.org/x/sys v0.0.0-20210525143221-35b2ab0089ea // indirect
 	golang.org/x/text v0.3.6
 	gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c // indirect

go.sum

@@ -4,6 +4,9 @@ github.com/ajg/form v1.5.1 h1:t9c7v8JUKu/XxOGBU0yjNpaMloxGEJhUkqFRq0ibGeU=
 github.com/ajg/form v1.5.1/go.mod h1:uL1WgH+h2mgNtvBq0339dVnzXdBETtL2LeUXaIv25UY=
 github.com/andybalholm/brotli v1.0.0 h1:7UCwP93aiSfvWpapti8g88vVVGp2qqtGyePsSuDafo4=
 github.com/andybalholm/brotli v1.0.0/go.mod h1:loMXtMfwqflxFJPmdbJO0a3KNoPuLBgiu3qAvBg8x/Y=
+github.com/asaskevich/govalidator v0.0.0-20210307081110-f21760c49a8d/go.mod h1:WaHUgvxTVq04UNunO+XhnAqY/wQc+bxr74GqbsZ/Jqw=
+github.com/aymerick/douceur v0.2.0 h1:Mv+mAeH1Q+n9Fr+oyamOlAkUNPWPlA8PPGR0QAaYuPk=
+github.com/aymerick/douceur v0.2.0/go.mod h1:wlT5vV2O3h55X9m7iVYN0TBM0NH/MmbLnd30/FjWUq4=
 github.com/boj/redistore v0.0.0-20180917114910-cd5dcc76aeff/go.mod h1:+RTT1BOk5P97fT2CiHkbFQwkK3mjsFAP6zCYV2aXtjw=
 github.com/bradfitz/gomemcache v0.0.0-20190329173943-551aad21a668/go.mod h1:H0wQNHz2YrLsuXOZozoeDmnHXkNCRmMW0gwFWDfEZDA=
 github.com/bradleypeabody/gorilla-sessions-memcache v0.0.0-20181103040241-659414f458e1/go.mod h1:dkChI7Tbtx7H1Tj7TqGSZMOeGpMP5gLHtjroHd4agiI=
@@ -151,6 +154,8 @@ github.com/gopherjs/gopherjs v0.0.0-20181017120253-0766667cb4d1 h1:EGx4pi6eqNxGa
 github.com/gopherjs/gopherjs v0.0.0-20181017120253-0766667cb4d1/go.mod h1:wJfORRmW1u3UXTncJ5qlYoELFm8eSnnEO6hX4iZ3EWY=
 github.com/gorilla/context v1.1.1 h1:AWwleXJkX/nhcU9bZSnZoi3h/qGYqQAGhq6zZe/aQW8=
 github.com/gorilla/context v1.1.1/go.mod h1:kBGZzfjB9CEq2AlWe17Uuf7NDRt0dE0s8S51q0aT7Yg=
+github.com/gorilla/css v1.0.0 h1:BQqNyPTi50JCFMTw/b67hByjMVXZRwGha6wxVGkeihY=
+github.com/gorilla/css v1.0.0/go.mod h1:Dn721qIggHpt4+EFCcTLTU/vk5ySda2ReITrtgBl60c=
 github.com/gorilla/securecookie v1.1.1 h1:miw7JPhV+b/lAHSXz4qd/nN9jRiAFV5FwjeKyCS8BvQ=
 github.com/gorilla/securecookie v1.1.1/go.mod h1:ra0sb63/xPlUeL+yeDciTfxMRAA+MP+HVt/4epWDjd4=
 github.com/gorilla/sessions v1.1.1/go.mod h1:8KCfur6+4Mqcc6S0FEfKuN15Vl5MgXW92AE8ovaJD0w=
@@ -194,6 +199,8 @@ github.com/mattn/go-isatty v0.0.12/go.mod h1:cbi8OIDigv2wuxKPP5vlRcQ1OAZbq2CE4Ky
 github.com/mattn/go-isatty v0.0.13 h1:qdl+GuBjcsKKDco5BsxPJlId98mSWNKqYA+Co0SC1yA=
 github.com/mattn/go-isatty v0.0.13/go.mod h1:cbi8OIDigv2wuxKPP5vlRcQ1OAZbq2CE4Kysco4FUpU=
 github.com/memcachier/mc v2.0.1+incompatible/go.mod h1:7bkvFE61leUBvXz+yxsOnGBQSZpBSPIMUQSmmSHvuXc=
+github.com/microcosm-cc/bluemonday v1.0.15 h1:J4uN+qPng9rvkBZBoBb8YGR+ijuklIMpSOZZLjYpbeY=
+github.com/microcosm-cc/bluemonday v1.0.15/go.mod h1:ZLvAzeakRwrGnzQEvstVzVt3ZpqOF2+sdFr0Om+ce30=
 github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
 github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd h1:TRLaZ9cD/w8PVh93nsPXa1VrQ6jlwL5oN8l14QlcNfg=
 github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
@@ -368,8 +375,8 @@ golang.org/x/net v0.0.0-20201010224723-4f7140c49acb/go.mod h1:sp8m0HH+o8qH0wwXwY
 golang.org/x/net v0.0.0-20201021035429-f5854403a974/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
 golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
 golang.org/x/net v0.0.0-20210428140749-89ef3d95e781/go.mod h1:OJAsFXCWl8Ukc7SiCT/9KSuxbyM7479/AVlXFRxuMCk=
-golang.org/x/net v0.0.0-20210525063256-abc453219eb5 h1:wjuX4b5yYQnEQHzd+CBcrcC6OVR2J1CN6mUy0oSxIPo=
+golang.org/x/net v0.0.0-20210614182718-04defd469f4e h1:XpT3nA5TvE525Ne3hInMh6+GETgn27Zfm9dxsThnX2Q=
-golang.org/x/net v0.0.0-20210525063256-abc453219eb5/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
+golang.org/x/net v0.0.0-20210614182718-04defd469f4e/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
 golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
 golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
 golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=

internal/processing/account/create.go

@@ -23,6 +23,7 @@
 
 	apimodel "github.com/superseriousbusiness/gotosocial/internal/api/model"
 	"github.com/superseriousbusiness/gotosocial/internal/gtsmodel"
+	"github.com/superseriousbusiness/gotosocial/internal/util"
 	"github.com/superseriousbusiness/oauth2/v4"
 )
 
@@ -44,7 +45,7 @@ func (p *processor) Create(applicationToken oauth2.TokenInfo, application *gtsmo
 	}
 
 	l.Trace("creating new username and account")
-	user, err := p.db.NewSignup(form.Username, reason, p.config.AccountsConfig.RequireApproval, form.Email, form.Password, form.IP, form.Locale, application.ID)
+	user, err := p.db.NewSignup(form.Username, util.RemoveHTML(reason), p.config.AccountsConfig.RequireApproval, form.Email, form.Password, form.IP, form.Locale, application.ID)
 	if err != nil {
 		return nil, fmt.Errorf("error creating new signup in the database: %s", err)
 	}

internal/processing/account/update.go

@@ -50,7 +50,8 @@ func (p *processor) Update(account *gtsmodel.Account, form *apimodel.UpdateCrede
 		if err := util.ValidateDisplayName(*form.DisplayName); err != nil {
 			return nil, err
 		}
-		if err := p.db.UpdateOneByID(account.ID, "display_name", *form.DisplayName, &gtsmodel.Account{}); err != nil {
+		displayName := util.RemoveHTML(*form.DisplayName) // no html allowed in display name
+		if err := p.db.UpdateOneByID(account.ID, "display_name", displayName, &gtsmodel.Account{}); err != nil {
 			return nil, err
 		}
 	}
@@ -59,7 +60,8 @@ func (p *processor) Update(account *gtsmodel.Account, form *apimodel.UpdateCrede
 		if err := util.ValidateNote(*form.Note); err != nil {
 			return nil, err
 		}
-		if err := p.db.UpdateOneByID(account.ID, "note", *form.Note, &gtsmodel.Account{}); err != nil {
+		note := util.SanitizeHTML(*form.Note) // html OK in note but sanitize it
+		if err := p.db.UpdateOneByID(account.ID, "note", note, &gtsmodel.Account{}); err != nil {
 			return nil, err
 		}
 	}

internal/processing/admin/createdomainblock.go

@@ -28,6 +28,7 @@
 	"github.com/superseriousbusiness/gotosocial/internal/gtserror"
 	"github.com/superseriousbusiness/gotosocial/internal/gtsmodel"
 	"github.com/superseriousbusiness/gotosocial/internal/id"
+	"github.com/superseriousbusiness/gotosocial/internal/util"
 )
 
 func (p *processor) DomainBlockCreate(account *gtsmodel.Account, domain string, obfuscate bool, publicComment string, privateComment string, subscriptionID string) (*apimodel.DomainBlock, gtserror.WithCode) {
@@ -51,8 +52,8 @@ func (p *processor) DomainBlockCreate(account *gtsmodel.Account, domain string,
 			ID:                 blockID,
 			Domain:             domain,
 			CreatedByAccountID: account.ID,
-			PrivateComment:     privateComment,
+			PrivateComment:     util.RemoveHTML(privateComment),
-			PublicComment:      publicComment,
+			PublicComment:      util.RemoveHTML(publicComment),
 			Obfuscate:          obfuscate,
 			SubscriptionID:     subscriptionID,
 		}

internal/processing/instance.go

@@ -60,7 +60,7 @@ func (p *processor) InstancePatch(form *apimodel.InstanceSettingsUpdateRequest)
 		if err := util.ValidateSiteTitle(*form.Title); err != nil {
 			return nil, gtserror.NewErrorBadRequest(err, fmt.Sprintf("site title invalid: %s", err))
 		}
-		i.Title = *form.Title
+		i.Title = util.RemoveHTML(*form.Title) // don't allow html in site title
 	}
 
 	// validate & update site contact account if it's set on the form
@@ -110,7 +110,7 @@ func (p *processor) InstancePatch(form *apimodel.InstanceSettingsUpdateRequest)
 		if err := util.ValidateSiteShortDescription(*form.ShortDescription); err != nil {
 			return nil, gtserror.NewErrorBadRequest(err, err.Error())
 		}
-		i.ShortDescription = *form.ShortDescription
+		i.ShortDescription = util.SanitizeHTML(*form.ShortDescription) // html is OK in site description, but we should sanitize it
 	}
 
 	// validate & update site description if it's set on the form
@@ -118,7 +118,7 @@ func (p *processor) InstancePatch(form *apimodel.InstanceSettingsUpdateRequest)
 		if err := util.ValidateSiteDescription(*form.Description); err != nil {
 			return nil, gtserror.NewErrorBadRequest(err, err.Error())
 		}
-		i.Description = *form.Description
+		i.Description = util.SanitizeHTML(*form.Description) // html is OK in site description, but we should sanitize it
 	}
 
 	// validate & update site terms if it's set on the form
@@ -126,7 +126,7 @@ func (p *processor) InstancePatch(form *apimodel.InstanceSettingsUpdateRequest)
 		if err := util.ValidateSiteTerms(*form.Terms); err != nil {
 			return nil, gtserror.NewErrorBadRequest(err, err.Error())
 		}
-		i.Terms = *form.Terms
+		i.Terms = util.SanitizeHTML(*form.Terms) // html is OK in site terms, but we should sanitize it
 	}
 
 	// process avatar if provided

internal/processing/media/create.go

@@ -26,6 +26,7 @@
 
 	apimodel "github.com/superseriousbusiness/gotosocial/internal/api/model"
 	"github.com/superseriousbusiness/gotosocial/internal/gtsmodel"
+	"github.com/superseriousbusiness/gotosocial/internal/util"
 )
 
 func (p *processor) Create(account *gtsmodel.Account, form *apimodel.AttachmentRequest) (*apimodel.Attachment, error) {
@@ -53,7 +54,7 @@ func (p *processor) Create(account *gtsmodel.Account, form *apimodel.AttachmentR
 	// TODO: handle this inside mediaHandler.ProcessAttachment (just pass more params to it)
 
 	// first description
-	attachment.Description = form.Description
+	attachment.Description = util.RemoveHTML(form.Description) // remove any HTML from the image description
 
 	// now parse the focus parameter
 	focusx, focusy, err := parseFocus(form.Focus)

internal/processing/media/update.go

@@ -26,6 +26,7 @@
 	"github.com/superseriousbusiness/gotosocial/internal/db"
 	"github.com/superseriousbusiness/gotosocial/internal/gtserror"
 	"github.com/superseriousbusiness/gotosocial/internal/gtsmodel"
+	"github.com/superseriousbusiness/gotosocial/internal/util"
 )
 
 func (p *processor) Update(account *gtsmodel.Account, mediaAttachmentID string, form *apimodel.AttachmentUpdateRequest) (*apimodel.Attachment, gtserror.WithCode) {
@@ -43,7 +44,7 @@ func (p *processor) Update(account *gtsmodel.Account, mediaAttachmentID string,
 	}
 
 	if form.Description != nil {
-		attachment.Description = *form.Description
+		attachment.Description = util.RemoveHTML(*form.Description)
 		if err := p.db.UpdateByID(mediaAttachmentID, attachment); err != nil {
 			return nil, gtserror.NewErrorInternalError(fmt.Errorf("database error updating description: %s", err))
 		}

internal/processing/status/create.go

@@ -29,7 +29,7 @@ func (p *processor) Create(account *gtsmodel.Account, application *gtsmodel.Appl
 		Local:                    true,
 		AccountID:                account.ID,
 		AccountURI:               account.URI,
-		ContentWarning:           form.SpoilerText,
+		ContentWarning:           util.RemoveHTML(form.SpoilerText),
 		ActivityStreamsType:      gtsmodel.ActivityStreamsNote,
 		Sensitive:                form.Sensitive,
 		Language:                 form.Language,

internal/processing/status/util.go

@@ -264,6 +264,10 @@ func (p *processor) processContent(form *apimodel.AdvancedStatusCreateForm, acco
 	// replace newlines with breaks
 	content = strings.ReplaceAll(content, "\n", "<br />")
 
-	status.Content = content
+	// sanitize html to remove any dodgy scripts or other disallowed elements
+	clean := util.SanitizeHTML(content)
+
+	// set the content as the shiny clean parsed content
+	status.Content = clean
 	return nil
 }

internal/util/sanitize.go

@@ -0,0 +1,50 @@
+/*
+   GoToSocial
+   Copyright (C) 2021 GoToSocial Authors admin@gotosocial.org
+
+   This program is free software: you can redistribute it and/or modify
+   it under the terms of the GNU Affero General Public License as published by
+   the Free Software Foundation, either version 3 of the License, or
+   (at your option) any later version.
+
+   This program is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+   GNU Affero General Public License for more details.
+
+   You should have received a copy of the GNU Affero General Public License
+   along with this program.  If not, see <http://www.gnu.org/licenses/>.
+*/
+
+package util
+
+import (
+	"github.com/microcosm-cc/bluemonday"
+)
+
+// '[A]llows a broad selection of HTML elements and attributes that are safe for user generated content.
+// Note that this policy does not allow iframes, object, embed, styles, script, etc.
+// An example usage scenario would be blog post bodies where a variety of formatting is expected along with the potential for TABLEs and IMGs.'
+//
+// Source: https://github.com/microcosm-cc/bluemonday#usage
+var regular *bluemonday.Policy = bluemonday.UGCPolicy().
+	RequireNoReferrerOnLinks(true).
+	RequireNoFollowOnLinks(true).
+	RequireCrossOriginAnonymous(true)
+
+// '[C]an be thought of as equivalent to stripping all HTML elements and their attributes as it has nothing on its allowlist.
+// An example usage scenario would be blog post titles where HTML tags are not expected at all
+// and if they are then the elements and the content of the elements should be stripped. This is a very strict policy.'
+//
+// Source: https://github.com/microcosm-cc/bluemonday#usage
+var strict *bluemonday.Policy = bluemonday.StrictPolicy()
+
+// SanitizeHTML cleans up HTML in the given string, allowing through only safe HTML elements.
+func SanitizeHTML(in string) string {
+	return regular.Sanitize(in)
+}
+
+// RemoveHTML removes all HTML from the given string.
+func RemoveHTML(in string) string {
+	return strict.Sanitize(in)
+}