Commit graph

1800 commits

Author SHA1 Message Date
tobi 68c8fe67cc
[chore/bugfix] Little DB fixes (#2726) 2024-03-06 15:40:37 +01:00
tobi b22e213e15
[feature/chore] Add Move database functions + cache (#2647)
* [feature/chore] Add Move database functions + cache

* add move mem ratio to envparsing.sh

* update comment
2024-03-06 11:18:57 +01:00
Vyr Cossont 61a2b91f45
[feature] Filters v1 (#2594)
* Implement client-side v1 filters

* Exclude linter false positives

* Update test/envparsing.sh

* Fix minor Swagger, style, and Bun usage issues

* Regenerate Swagger

* De-generify filter keywords

* Remove updating filter statuses

This is an operation that the Mastodon v2 filter API doesn't actually have, because filter statuses, unlike keywords, don't have options: the only info they contain is the status ID to be filtered.

* Add a test for filter statuses specifically

* De-generify filter statuses

* Inline FilterEntry

* Use vertical style for Bun operations consistently

* Add comment on Filter DB interface

* Remove GoLand linter control comments

Our existing linters should catch these, or they don't matter very much

* Reduce memory ratio for filters
2024-03-06 11:15:58 +01:00
dependabot[bot] 7bc536d1f7
[chore]: Bump github.com/stretchr/testify from 1.8.4 to 1.9.0 (#2714)
* [chore]: Bump github.com/stretchr/testify from 1.8.4 to 1.9.0

Bumps [github.com/stretchr/testify](https://github.com/stretchr/testify) from 1.8.4 to 1.9.0.
- [Release notes](https://github.com/stretchr/testify/releases)
- [Commits](https://github.com/stretchr/testify/compare/v1.8.4...v1.9.0)

---
updated-dependencies:
- dependency-name: github.com/stretchr/testify
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* cast the exxpected test values as float32 to bypass (possible) testify suite regression

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kim <grufwub@gmail.com>
2024-03-04 15:05:24 +00:00
kim d85727e184
[bugfix] check remote status permissibility (#2703)
* add more stringent checks for remote status permissibility

* add check for inreplyto of a remote status being a boost

* do not permit inReplyTo boost wrapper statuses

* change comment wording

* fix calls to NewFederator()

* add code comments for NotPermitted() and SetNotPermitted()

* improve comment

* check that existing != nil before attempting delete

* ensure replying account isn't suspended

* use a debug log instead of info. check for boost using ID

* shorten log string length. make info level

* add note that replying to boost wrapper status shouldn't be able to happen anyways

* update to use onFail() function
2024-03-04 12:30:12 +00:00
tobi f487fc5d4b
[bugfix] Sanitize incoming PropertyValue fields (#2722) 2024-03-04 11:46:59 +01:00
tobi 0b35257312
[docs] Update HTTP signature docs a bit (#2721)
* [docs] Update HTTP signature docs a bit

* Update docs/federation/federating_with_gotosocial.md

Co-authored-by: Daenney <daenney@users.noreply.github.com>

* Update docs/federation/federating_with_gotosocial.md

Co-authored-by: Daenney <daenney@users.noreply.github.com>

---------

Co-authored-by: Daenney <daenney@users.noreply.github.com>
2024-03-04 11:27:11 +01:00
Milas Bowman 66d9297e64
[bugfix/tracing] fix broken tracing due to conflicting schema url (#2712)
The OpenTelemetry SDK is very strict about the schema version when
the `Resource` is initialized.

Specifically, different schema versions _CANNOT_ be mixed, and since
the default SDK resource (which is merged with the user-defined one)
defines a schema URL, the `semconv` imports are really prone to being
out-of-sync.

The best way to avoid this is to merge a _schemaless_ resource. This
is fine...there's plenty of other ways to get `semconv` out of sync,
and the core service attributes (e.g. `service.name`) should not ever
change.

Additionally, any errors here are now propagated so that they'll be
visible instead of silently swallowed.
2024-03-04 11:10:15 +01:00
dependabot[bot] c7845c70d6
[chore]: Bump golang.org/x/crypto from 0.19.0 to 0.20.0 (#2716) 2024-03-04 09:42:28 +00:00
dependabot[bot] adb4cdcf6c
[chore]: Bump modernc.org/sqlite from 1.28.0 to 1.29.2 (#2718) 2024-03-04 09:42:11 +00:00
kim ea0efb8094
[bugfix] update postgresqlstmt to correctly use postgres err hook (#2711) 2024-03-03 11:21:25 +01:00
Daenney 40ba93654e
[docs] Use human sizes for size limitations (#2706)
This switches the documentation to use human sizes, so 50MiB instead of
an integer number of bytes. This makes it much easier to understand what
values are set, and less likely to accidentally set the wrong value.
2024-03-01 14:47:03 +00:00
kim fcecd0c952
[bugfix] unwrap boosts when checking in-reply-to status (#2702)
* add stronger checks on status being replied to

* update error code test is expecting
2024-02-29 15:20:57 +01:00
Vyr Cossont c2a691fd83
[chore] Disable the syslog long message over Unix datagram socket test on macOS (#2700) 2024-02-29 13:25:15 +01:00
tobi 504c4f227f
[chore] re-add statuses_account_id_id_idx if removed (#2699)
* [chore] re-add `statuses_account_id_id_idx` if removed

* if not exists
2024-02-28 12:53:46 +01:00
tobi f29d429699
[chore] add log line about lengthy reindex migration (#2695) 2024-02-27 18:46:48 +01:00
Vyr Cossont ad28b9f166
[bugfix] Account timeline: exclude self-replies that mention other accounts (#2670)
* Account timeline: exclude self-replies that mention other accounts

* Add index for querying unmentioned statuses

* remove now unused statuses_account_id_id_idx

---------

Co-authored-by: tobi <tobi.smethurst@protonmail.com>
2024-02-27 18:18:40 +01:00
kim 90dcc008c5
[feature] add script to test import / export cycle of a gotosocial instance (#2693)
* add script to test import / export cycle of a gotosocial instance

* use suffix for setting mktemp file suffix
2024-02-27 16:29:56 +01:00
9p4 9bf448be7a
[feature/oidc] Add support for very basic RBAC (#2642)
* Add support for very basic RBAC

* Add some small tests for allowedGroup and adminGroup

* Switch to table-driven tests
2024-02-27 16:07:29 +01:00
tobi feb6abbab2
[chore/docs] Various little docs updates (#2691)
* [chore/docs] Various little docs updates

* Update docs/api/swagger.md

Co-authored-by: Daenney <daenney@users.noreply.github.com>

---------

Co-authored-by: kim <89579420+NyaaaWhatsUpDoc@users.noreply.github.com>
Co-authored-by: Daenney <daenney@users.noreply.github.com>
2024-02-27 14:39:22 +00:00
Daenney 6528592dd2
[feature] Block Amazonbot (#2692)
Blocks the Amazon crawler bot.

Closes: #2686
2024-02-27 13:25:08 +00:00
tobi 9cadc764b3
[feature] Add experimental instance-federation-spam-filter option (#2685)
* [chore] Move `visibility` to `filter/visibility`

* [feature] Add experimental instance-federation-spam-filter option
2024-02-27 12:22:05 +00:00
dependabot[bot] 87e8cca3ae
[chore]: Bump github.com/tdewolff/minify/v2 from 2.20.17 to 2.20.18 (#2689)
Bumps [github.com/tdewolff/minify/v2](https://github.com/tdewolff/minify) from 2.20.17 to 2.20.18.
- [Release notes](https://github.com/tdewolff/minify/releases)
- [Commits](https://github.com/tdewolff/minify/compare/v2.20.17...v2.20.18)

---
updated-dependencies:
- dependency-name: github.com/tdewolff/minify/v2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-02-26 10:32:02 +00:00
tux93 d8956d710e
[docs/bugfix] Allow access to TMP directories in example AppArmor config (#2683)
* Remove trailing whitespace from example config

* Update and extend example AppArmor profile
2024-02-24 09:44:53 +01:00
Camille Frantz 403f5c0528
[docs] Add oauth token authentication workflow (#2655)
* add authentication workflow to docs

* fixed the markdown callouts and added the new page to the mkdocs.yml

* fixed the markdown callouts properly this time

* applied the suggested edits

* inline styles don't render in code blocks; added bash to the curl code blocks

* scopes list was not rendering as a list; fixed a typo in getting your token warning

* scopes can be a space separated list and not just one

* apply suggested change to authentication.md

Co-authored-by: Daenney <daenney@users.noreply.github.com>

* applied latest suggested edits

* added information on Gts scoped tokens

* fixed admonition formatting

---------

Co-authored-by: Daenney <daenney@users.noreply.github.com>
2024-02-24 09:41:38 +01:00
tobi 4b0eefbcc9
[chore] Increase default max image description to 1500 chars, collapse cw char count into status (#2682)
* [chore] Make default max image description 1500 chars, collapse cw char count into status

* oops

* tests
2024-02-23 18:28:09 +00:00
kim 1d51e3c8d6
[bugfix] 2643 bug search for account url doesnt always work when redirected (#2673)
* update activity library so dereferencer returns full response and checks *final* link to allow for redirects

* temporarily add bodged fixed library

* remove unused code

* update getAccountFeatured() to use dereferenceCollectionPage()

* make sure to release map

* perform a 2nd decode to ensure reader is empty after primary decode

* add comment explaining choice of using Decode() instead of Unmarshal()

* update embedded activity library to latest matching https://github.com/superseriousbusiness/activity/pull/21

* add checks to look for changed URI and re-check database if redirected

* update max iteration count to 512, add checks during dereferenceAncestors() for indirect URLs

* remove doubled-up code

* fix use of status instead of current

* use URIs for checking equality for security

* use the latest known URI for boost_of_uri in case original was an indirect

* add dereferenceCollection() function for dereferenceAccountFeatured()

* pull in latest github.com/superseriousbusiness/activity version (and remove the bodge!!)

* fix typo in code comments

* update decodeType() to accept a readcloser and handle body closing

* switch to checking using BoostOfID and add note why not using BoostOfURI

* ensure InReplyTo gets unset when deleting status parent in case currently stubbed

* add tests for Collection and CollectionPage iterators
2024-02-23 16:24:40 +01:00
tobi 37a39b98ce
[feature] Add "what is this" section to index template (#2680) 2024-02-23 15:08:32 +00:00
tobi 5aca09b59e
[chore] Rename frontend.tmpl to settings.tmpl, remove unused "lightgray" class (#2674) 2024-02-21 16:11:11 +01:00
tobi 2d9e498f96
[bugfix] Fix wide images being squished when used as instance avatar (#2669) 2024-02-21 14:24:04 +01:00
kim 291e180990
[bugfix] fix possible mutex lockup during streaming code (#2633)
* rewrite Stream{} to use much less mutex locking, update related code

* use new context for the stream context

* ensure stream gets closed on return of writeTo / readFrom WSConn()

* ensure stream write timeout gets cancelled

* remove embedded context type from Stream{}, reformat log messages for consistency

* use c.Request.Context() for context passed into Stream().Open()

* only return 1 boolean, fix tests to expect multiple stream types in messages

* changes to ping logic

* further improved ping logic

* don't export unused function types, update message sending to only include relevant stream type

* ensure stream gets closed 🤦

* update to error log on failed json marshal (instead of panic)

* inverse websocket read error checking to _ignore_ expected close errors
2024-02-20 18:07:49 +00:00
tobi 8cafa6b74b
[feature] Add requested_by to relationship model (#2672)
* [feature] Add `requested_by` to relationship model

* whoops, missed some tests
2024-02-20 17:50:54 +00:00
tobi 65a273bc39
[bugfix] use start + end line in regex when validating emoji via API (#2671) 2024-02-20 11:46:04 +01:00
tobi ddfc725744
[chore/frontend] Use different background color for block quotes (#2668) 2024-02-19 16:51:57 +01:00
tobi cc09f612ec
[chore/frontend] Don't use italics for block quotes (#2667) 2024-02-19 16:34:45 +01:00
tobi d10226e912
[bugfix] Use ptr for instance stats entries to avoid skipping 0 values (#2666)
* [bugfix] Use ptr for instance stats entries to avoid skipping 0 values

* comment explaining why stats values are pointers
2024-02-19 13:17:14 +01:00
tobi 0554550acb
[bugfix] Ensure local statuses always get a threadID so they can be muted (#2665)
* [chore/bugfix] Ensure threadID always set on local statuses

* test
2024-02-19 11:50:49 +01:00
dependabot[bot] e2ebcbb516
[chore]: Bump github.com/jackc/pgx/v5 from 5.5.2 to 5.5.3 (#2664)
Bumps [github.com/jackc/pgx/v5](https://github.com/jackc/pgx) from 5.5.2 to 5.5.3.
- [Changelog](https://github.com/jackc/pgx/blob/master/CHANGELOG.md)
- [Commits](https://github.com/jackc/pgx/compare/v5.5.2...v5.5.3)

---
updated-dependencies:
- dependency-name: github.com/jackc/pgx/v5
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-02-19 11:20:15 +01:00
Milas Bowman af1a26a68f
[feature] Add Mastodon-compatible HTTP signature fallback (#2659)
On outgoing `GET` requests that are signed (e.g. authorized fetch),
if the initial request fails with `401`, try again, but _without_
the query parameters included in the HTTP signature.

This is primarily useful for compatibility with Mastodon; though
hopefully this can be removed in the not-too-distant future, as
they've started changing their behavior here.

Signed-off-by: Milas Bowman <devnull@milas.dev>
2024-02-19 11:18:17 +01:00
dependabot[bot] 2db115fa36
[chore]: Bump github.com/minio/minio-go/v7 from 7.0.66 to 7.0.67 (#2662)
Bumps [github.com/minio/minio-go/v7](https://github.com/minio/minio-go) from 7.0.66 to 7.0.67.
- [Release notes](https://github.com/minio/minio-go/releases)
- [Commits](https://github.com/minio/minio-go/compare/v7.0.66...v7.0.67)

---
updated-dependencies:
- dependency-name: github.com/minio/minio-go/v7
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-02-19 09:36:50 +00:00
dependabot[bot] 30f6e6c7fe
[chore]: Bump github.com/tdewolff/minify/v2 from 2.20.16 to 2.20.17 (#2661) 2024-02-19 07:23:08 +00:00
tobi a314d2b539
[bugfix] Refactor parse mention, fix local mention bug (#2657)
* [bugfix] Refactor parse mention, fix local mention bug

* originAccount -> originAcct
2024-02-19 03:48:20 +00:00
tobi b14cd645f9
[chore] Comment out silly, frequently-failing GetStatusTwice test (#2656)
* [chore] Comment out silly, frequently-failing `GetStatusTwice` test

* lord
2024-02-18 11:04:02 +01:00
tobi 40f9eefc21
[bugfix] Fix dereferencing ancestors on new status create (#2652)
* [bugfix] Pass `latest` to dereferenceThread instead of barebones status

* only mark status orphaned if visibility suggests parent is really deleted

* tone down "not deref'd" warnings, since they represent a legit visibility situation

* remove FAQ entry for "status not deref'd yet"
2024-02-18 09:49:40 +00:00
tobi 83a4adbd95
[chore] Disable Move API endpoints for now until Move is fully implemented in the backend (#2650)
* [chore] Disable Move API endpoints for now until Move is fully implemented in the backend

* disable other form fields

* clarify that moving to GtS also isn't implemented yet
2024-02-17 17:03:44 +01:00
tobi d3f35e8eba
[chore] refactor extractFromCtx a bit (#2646) 2024-02-17 15:20:39 +01:00
Daenney a41e8a95bf
[chore] Simplify the User-Agent string (#2645)
* [chore] Simplify the User-Agent string

RFC 9110[1] includes a definition for the format of a user-agent header:

User-Agent = product *( RWS ( product / comment ) )
             product         = token ["/" product-version]
             product-version = token
	     comment        = "(" *( ctext / quoted-pair / comment ) ")"
             ctext          = HTAB / SP / %x21-27 / %x2A-5B / %x5D-7E / obs-text

An example given in the RFC: User-Agent: CERN-LineMode/2.15 libwww/2.17b3

The idea is typically start with the most important product/version, add
a (comment) if necessary and then include any auxilliary products.
However, the RFC warns against including too many auxiliary products as
those can be unnecessarily revealing.

For automated systems (i.e not a browser), the common and recommended
format is <product></version> (+uri-for-contact), followed with any
additional <product>/<version> pairs that are relevant.

This changes our UA to match that convention more closely. This makes it
easier for administrators who do user-agent parsing for statistics or
other purposes to correctly identify the version of GoToSocial.
Currently tools tend to get confused by the lack of a /<version> on the
start of our string.

[1]: https://www.rfc-editor.org/rfc/rfc9110.html#name-user-agents

* [chore] Don't use app name in UA

From all the GtS UAs I've collected, nobody seems to set/change this,
so we might as well use the static string.

The main usefulness for this is when you have multilpe GtS instances
connecting to the same DB, so they can identify as different instances
by changing the application name (though it should already be obvious
from having different usernames).
2024-02-17 09:54:10 +01:00
kim 8b8211986e
[chore] also allow text/xml in place of application/xml (#2640) 2024-02-14 13:07:58 +01:00
kim 2bafd7daf5
[bugfix] add stricter checks during all stages of dereferencing remote AS objects (#2639)
* add stricter checks during all stages of dereferencing remote AS objects

* a comment
2024-02-14 12:13:38 +01:00
Tsuribori 142b7ec54f
[feature] Add metrics for instance user count, statuses count and federating instances count (#2592)
Co-authored-by: Tsuribori <none@example.org>
2024-02-14 11:58:55 +01:00