Commit graph

1227 commits

Author SHA1 Message Date
kernelmethod 1652633d93
[docs] Add AppArmor profile for Debian and Ubuntu installations (#1183)
* Enable the 'admonitions' Markdown extension for Mkdocs.

The admonitions extension to Python-Markdown allows you to include
rST-style "admonitions" to Markdown documents, for instance,

    !!! note
        Here's an important note to keep in mind!

In general, the current documentation uses bold text to try to achieve
the same effect, which is a bit harder to notice and makes it difficult
to differentiate between "here's something useful to know" versus "here
there be dragons".

* Add AppArmor profile and documentation for LSM-related sandboxing

This commit adds an AppArmor profile for gotosocial in
examples/apparmor/gotosocial. This will (hopefully) serve as a helpful
security mitigation for people are planning on deploying GTS on a
Debian-family Linux distribution.

I've also updates the documentation to include some information about
deploying GTS with either AppArmor or SELinux (moving the documentation
for the former out of the "binary installation guide" docs).
2022-11-30 23:09:26 +01:00
tobi 3a11861ac6
[bugfix] Don't call strings.ToLower() on usernames when selecting account by domain+username (#1190)
* don't lowercase account username when doing a select

* test getting remote user with uppercase username
2022-11-30 17:44:02 +01:00
tobi 8d581deb28
[bugfix] use correct key for PATCHing admin email (#1189) 2022-11-30 16:41:27 +01:00
tobi 927117d8e3
[bugfix] Avoid accidentally marking changed emojis as orphaned + pruning them (#1188)
* add predictable instance account to tests, use it in emoji urls + paths

* use static image url to select emojis when pruning orphaned
2022-11-30 16:20:57 +01:00
f0x52 a1dda22672
[bugfix] change page width to prevent scrollbar overflow (#1182) 2022-11-30 13:15:33 +01:00
kim cfa8d7900c
[bugfix] don't return error during account serialize on deleted emoji (#1177)
* only return error for emoji fetch if NOT errnoentries

Signed-off-by: kim <grufwub@gmail.com>

* reformat gts->api model slice conversion to standard error behaviours and reduce code reuse

Signed-off-by: kim <grufwub@gmail.com>

Signed-off-by: kim <grufwub@gmail.com>
2022-11-29 17:59:59 +00:00
kim d445c60a26
[bugfix] wrap the correct error on failed account update (#1176)
Signed-off-by: kim <grufwub@gmail.com>

Signed-off-by: kim <grufwub@gmail.com>
2022-11-29 18:40:30 +01:00
tobi 97f5453378
[chore] Tidy up some of the search logic (#1082)
* start refactoring some of the search + deref logic

* add tests for search api

* rename GetRemoteAccount + GetRemoteStatus

* make search function a bit simpler + clearer

* fix little fucky wucky uwu owo i'm just a little guy

* update faulty switch statements

* update test to use storage struct

* redo switches for clarity

* reduce repeated logic in search tests

* fastfail getstatus by uri

* debug log + trace log better

* add implementation note

* return early if no result for namestring search

* return + check on dereferencing error types

* errors hah what errors

* remove unneeded error type alias, add custom error text during stringification itself

* fix a woops recursion 🙈

Signed-off-by: kim <grufwub@gmail.com>
Co-authored-by: kim <grufwub@gmail.com>
2022-11-29 09:24:55 +00:00
tobi daf44ac2b7
[chore] Bump database dependencies (#1164)
github.com/uptrace/bun v1.1.8 -> v1.1.9
github.com/uptrace/bun/pgdialect v1.1.8 -> v1.1.9
github.com/uptrace/bun/sqlitedialect v1.1.8 -> v1.1.9
modernc.org/sqlite v1.18.2 -> v1.19.5
2022-11-28 11:19:39 +01:00
dependabot[bot] fe39d50e09
[chore]: Bump codeberg.org/gruf/go-store/v2 from 2.0.9 to 2.0.10 (#1160)
Bumps codeberg.org/gruf/go-store/v2 from 2.0.9 to 2.0.10.

---
updated-dependencies:
- dependency-name: codeberg.org/gruf/go-store/v2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-11-28 09:01:53 +00:00
kim 0c1b1b01f8
fix missing lookup cache key for invalid domain block (#1158) 2022-11-27 14:11:49 +00:00
Daniele Sluijters c534230600
[bugfix] Use Host domain for UA (#1156)
In d6f4d196c9 we swapped to use the
AccountDomain but that actually goes against the intent of the change.
This reverts that change and uses the host domain again.
2022-11-27 00:09:09 +00:00
Daniele Sluijters c5ae88c51b
[chore] Set User-Agent header in transport (#1154)
Currently requests set their own User-Agent. This moves it down to set
it in the transport's do() method, to guarantee it's always set on all
requests.
2022-11-26 20:19:42 +00:00
Daniele Sluijters d6f4d196c9
Tweak the User-Agent a bit (#1153)
* [bugfix] Use AccountDomain for user agent

By using the account domain we can pinpoint the source of the request
more accurately when looking at the User-Agent header.

* [chore] Align user-agent header with spec

Based on RFC 7231, our User-Agent header doesn't quite match. It seems
to always want Name [/ Version] pairs, with comments in parenthesis and
multiple comments separated by a semicolon.

Align our UA with that, using application name first by itself in case
someone has customised it with the source instance in a comment. Follow
that up with gotosocial/<version> and a comment pointing at the source
code.

This also drops the mention of gofed/activity since a fork is in use.
2022-11-26 20:15:19 +00:00
Daniele Sluijters 746f3fa4e6
Additional IP range validations (#1152)
* [bugfix] Ensure requests happen over TCP

It's possible for the network to be udp4 or udp6. This is rather
unlikely to occur, but since we're given the network anyway as part of
the Sanitize function getting called we might as well check for it.

* [chore] Align reserved v6 blocks to IANA registry

* [chore] Add test for ValidateIP

The net and netip packages diverge in that net.ParseIP will consider an
IPv4-mapped address to be an IPv4 address and as such it would get
caught by the IPv4Reserved list. However, netip considers it an IPv6
address, so we need to ensure the mapped range is in IPv6Reserved.

* [chore] Align reserved v4 blocks to IANA registry

This includes a number of tests for /32's explicitly called out in the
registry to ensure we always consider those invalid.
2022-11-26 11:09:55 +00:00
Daniele Sluijters e6cd81babc
[bugfix]: Fix IPv6 validation (#1150)
* [bugfix]: Fix IPv6 validation

The current code considers ff00::/8 valid, but contrary to the comment
that's not the global unicast range. ff-prefixes in IPv6 denote
multicast.

This adapts the code to take the same approach as IPv4, explicitly
blacklisting reserved internal/private ranges.

* [chore] Add missing 4 in IPv4Reserved doc comment
2022-11-25 23:28:03 +00:00
Phil Hagelberg 8b0c92ec41
[docs] Explain that before 0.6.0, account confirm was necessary. (#1149)
Since the documentation site only shows the latest version of the
docs, we need the docs to explain how to use the latest stable
release, not just the latest git version.
2022-11-25 18:56:01 +01:00
tobi 13e9abd02a
[feature] Add admin media prune orphaned CLI command (#1146)
* add FilePath regex

* add `admin media prune orphaned` command

* add prune orphaned function to media manager

* don't mark flag as required

* document admin media prune orphaned cmd

* oh envparsing.sh you coy minx
2022-11-25 17:23:42 +00:00
Phil Hagelberg 9e18c7f996
[bugfix] Change emailVerified to true for admin account create (#1140)
The NewSignup method was already being called with
requireApproval=false, but it had emailVerified=false as well, which
meant that it was required to use the `admin account confirm` command
to verify the email before the newly-created user could log in.

I think that was probably an oversight; effectively it did require
approval anyway. Changing emailVerified to true allows you to just
create the account and log in immediately, reducing the opportunity
for manual error to sneak in.

Also updated the docs to remove the mention of needing to confirm new
accounts. However, I've left the confirmation command alone because I
think once we have web signups, it will be needed in that context.
2022-11-25 16:44:27 +01:00
f0x52 5c5c8ceaf0
[chore] Fix frogend admin emoji patch (#1148)
* fix className

* fix unable to set category on new emoji

* wrap emoji groups
2022-11-25 16:37:57 +01:00
f0x52 665d902fd7
[feature/frogend] modify local emoji (#1143)
* update danger button red

* emoji category and image modification

* debug bundles in dev

* fix linting error
2022-11-25 15:49:48 +01:00
tobi b6dbe21026
[feature] PATCH /api/v1/admin/custom_emojis/{id} endpoint (#1061)
* start adding admin emoji PATCH stuff

* updating works OK, now how about copying

* allow emojis to be copied

* update swagger docs

* update admin processer to use non-interface storage driver

* remove shortcode updating for local emojis

* go fmt

Co-authored-by: f0x52 <f0x@cthu.lu>
2022-11-24 18:12:07 +00:00
tobi 3e82196d5e
[bugfix] Fix status boosts giving 404 (#1137)
We broke this at some point recently, and i'm not sure when.

In any case, i updated some of the logic in there + added a test for it.
2022-11-24 16:12:43 +01:00
tobi da8954858a
[bugfix] Prevent future statuses entering timelines (#1134)
* [bugfix] Prevent future statuses entering timeline
Statuses created more than 5 minutes into the future are now rejected in the visibility package.

* Come on buddy
2022-11-24 13:54:49 +01:00
kim fcb9c0bb8b
[chore] cleanup storage implementation, no need for multiple interface types (#1131)
Signed-off-by: kim <grufwub@gmail.com>

Signed-off-by: kim <grufwub@gmail.com>
2022-11-24 09:35:46 +01:00
tobi c9d893fec1
[feature/performance] Fail fast when doing remote transport calls inside incoming request contexts (#1119)
* [feature/performance] Fail fast when doing remote transport calls inside incoming request contexts

* [chore] Reduce outgoing request timeout to 15s

* log error messages when fastfailing

* use context.Value() instead of wrapped context, wrap error with fastfail instead of extra log entry

* add fast-fail context key test

Signed-off-by: kim <grufwub@gmail.com>
Co-authored-by: kim <grufwub@gmail.com>
2022-11-23 21:40:07 +00:00
dx 8942a70856
[docs] Clarify that reverse proxy setups need trusted-proxies (#1127)
* [docs] Clarify that reverse proxy setups need trusted-proxies

* Remove the jq usage
2022-11-23 16:10:56 +01:00
f0x52 5ba5fb3154
[chore/frogend] fix profile preview css, use role info (#1130)
* fix profile preview css, use role info

* remove unused redux selector
2022-11-23 16:10:02 +01:00
tobi 1281a9fd4a
[chore/frontend] remove references to unused bundle.js (#1128) 2022-11-23 15:32:57 +01:00
tobi 50dc179d33
[feature] Prune timelines once per hour to plug memory leak (#1117)
* export highest/lowest ULIDs as proper const

* add stop + start to timeline manager, other small fixes

* unexport unused interface funcs + tidy up

* add LastGot func

* add timeline Prune function

* test prune

* update lastGot
2022-11-22 18:38:10 +00:00
tobi 90bbcf1bcf
[docs] Document http/s/socks5 proxy use (#1118) 2022-11-22 17:51:51 +01:00
tobi 11cd60883d
[chore] fix broken buildx (whoops) (#1116) 2022-11-22 15:09:45 +01:00
kim a898160b0c
[chore] use kv.KVStore also for S3 storage (#1113)
* replace s3 storage implementation to also use kv.KVStore

Signed-off-by: kim <grufwub@gmail.com>

* pull in latest `go-store` fix

Signed-off-by: kim <grufwub@gmail.com>

* pull-in go-store v2.0.9 fixes, update s3 put chunk size to 5MiB

Signed-off-by: kim <grufwub@gmail.com>

Signed-off-by: kim <grufwub@gmail.com>
2022-11-22 14:28:55 +01:00
Forest Johnson 6d43319fe1
[docs] Document non-buildx cross compilation for docker image (#1115)
* Enable / document cross compilation for docker image

* remove sudo

* explain whats up with BUILDPLATFORM/TARGETPLATFORM
2022-11-22 12:17:41 +01:00
Norwin f88cb3aebc
[chore] markdown: disable Smartypants rendering (#1111)
* [enhancement] markdown: disable SmartyPantsFractions

fixes #1028

* some fractions are still converted to unicode

to fix that, we might need to disable smartypants indeed in its entirety

* disable smartypants completely

for lack of a better simple solution
2022-11-22 11:08:50 +01:00
Phil Hagelberg b153808472
[docs] Add first draft of FAQ (#1086)
* Clarify that gotosocial doesn't use recommendation algorithms

It uses lots of algorithms.

    ~/src/gotosocial $ rgrep algorithm . | wc
        856    8971  143395

* Add first draft of the FAQ.

* Suggested FAQ changes.

Mention backfill of posts as not yet being implemented; link to FAQ
from readme/index.

Mention that GtS does serve up some UI; just not most interaction.
2022-11-21 13:19:51 +01:00
kim 0c2971a567
[chore] update readme with kim's links (#1093)
* add kim's code and fediverse profiles

Signed-off-by: kim <grufwub@gmail.com>

* fix words

Signed-off-by: kim <grufwub@gmail.com>

* undo my accidental inclusion of custom version string...

Signed-off-by: kim <grufwub@gmail.com>

Signed-off-by: kim <grufwub@gmail.com>
2022-11-21 13:18:25 +01:00
dependabot[bot] 274626ab5e
[chore]: Bump github.com/minio/minio-go/v7 from 7.0.43 to 7.0.44 (#1107)
Bumps [github.com/minio/minio-go/v7](https://github.com/minio/minio-go) from 7.0.43 to 7.0.44.
- [Release notes](https://github.com/minio/minio-go/releases)
- [Commits](https://github.com/minio/minio-go/compare/v7.0.43...v7.0.44)

---
updated-dependencies:
- dependency-name: github.com/minio/minio-go/v7
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-11-21 13:17:12 +01:00
kim 4a9538593c
[chore] simplify logging to remove "safetywriter" given os.File has concurrency protection (#1083)
Signed-off-by: kim <grufwub@gmail.com>

Signed-off-by: kim <grufwub@gmail.com>
2022-11-20 17:40:15 +01:00
kim 2eea3d57e0
[chore] improve errors returned fetching account/statues from DB (#1084)
Signed-off-by: kim <grufwub@gmail.com>

Signed-off-by: kim <grufwub@gmail.com>
2022-11-20 17:38:58 +01:00
kim 5d55e8d920
[performance] add account block DB cache and remove block query joins (#1085)
* add account block DB cache and remove reliance on relational joins

* actually include cache key arguments...

* add a PutBlock() method which also updates the block cache, update tests accordingly

* use `PutBlock` instead of `Put(ctx, block)`

* add + use functions for deleting + invalidating blocks

Signed-off-by: kim <grufwub@gmail.com>
Co-authored-by: tsmethurst <tobi.smethurst@protonmail.com>
2022-11-20 16:33:49 +00:00
kim 9be16852f2
[bugfix] fix possible infinite loop on federated AP profile delete (#1091)
* refactor federator account statuses delete to better catch errors, ensure next maxID is always set

Signed-off-by: kim <grufwub@gmail.com>

* fix error statement missing 2nd format operator

Signed-off-by: kim <grufwub@gmail.com>

Signed-off-by: kim <grufwub@gmail.com>
2022-11-20 15:57:19 +01:00
tobi 0490440fe0
[docs] update forking instructions (#1080) 2022-11-20 15:50:11 +01:00
tobi be3d999c21
[chore] Little Dockerfile updates (#1071)
* [chore] Little Dockerfile updates

* update comment on USER directive (thanks chris)
2022-11-19 10:18:01 +01:00
Michael Manfre 67106c9dc4
[feature] Support markdown format for Account bio/note (#1037)
* [feature] Status format also controls bio format

* test
2022-11-19 10:07:51 +01:00
Leonora Tindall 2216f253c1
[docs] Move and document logs (#1076)
Rather than a single access log and no error logging, put the logs in
/var/log/gotosocial/, and document creating and owning that directory
2022-11-19 10:06:37 +01:00
Leonora Tindall c964856927
[docs] Remove trailing / from proxy_pass for nginx (#1077)
The trailing / can break relative URLs.
2022-11-19 10:05:31 +01:00
kim 45ae719bd9
[performance] replace account emojis relational query with separate calls to emojiDB to rely on cache (#1074)
Signed-off-by: kim <grufwub@gmail.com>

Signed-off-by: kim <grufwub@gmail.com>
2022-11-18 18:29:25 +01:00
kim dccc2eee81
[performance] replace status query relationals with separate calls in order to rely on caches more (#1073)
Signed-off-by: kim <grufwub@gmail.com>

Signed-off-by: kim <grufwub@gmail.com>
2022-11-18 18:28:33 +01:00
tobi d98a48b446
[performance] don't use relations to select notification structs, use caches instead (#1072) 2022-11-18 17:23:14 +01:00