* close pipereader on failed data function
* gently slurp the bytes
* readability updates
* go fmt
* tidy up file server tests + add more cases
* start moving io wrappers to separate iotools package. Remove use of buffering while piping recache stream
Signed-off-by: kim <grufwub@gmail.com>
* add license text
Signed-off-by: kim <grufwub@gmail.com>
Co-authored-by: kim <grufwub@gmail.com>
Lots of these were appearing:
```
*459 connect() failed (111: Connection refused) while connecting to upstream
```
This change resolves it, see https://stackoverflow.com/a/52550758
* for domain block lookups, lookup along subdomain parts
Signed-off-by: kim <grufwub@gmail.com>
* only lookup up to a max of 5 domain parts to prevent DOS, limit inserted domains to max of 5 subdomains
Signed-off-by: kim <grufwub@gmail.com>
* add test for domain block wildcarding
Signed-off-by: kim <grufwub@gmail.com>
* check cached status first, increase cached domain time
Signed-off-by: kim <grufwub@gmail.com>
* fix domain wildcard part building logic
Signed-off-by: kim <grufwub@gmail.com>
* create separate domain.BlockCache{} type to hold all domain blocks in memory
Signed-off-by: kim <grufwub@gmail.com>
* remove unused variable
Signed-off-by: kim <grufwub@gmail.com>
* add docs and test to domain block cache, check for domain == host in domain block getter funcs
Signed-off-by: kim <grufwub@gmail.com>
* add license text
Signed-off-by: kim <grufwub@gmail.com>
* check order in which we check primary cache
Signed-off-by: kim <grufwub@gmail.com>
* add better documentation of how domain block checking is performed
Signed-off-by: kim <grufwub@gmail.com>
* change
Signed-off-by: kim <grufwub@gmail.com>
Signed-off-by: kim <grufwub@gmail.com>
* simplify pruneRemote, remove unncecessary media trace logging, update RemoteOlderThan() to include headers/avis
Signed-off-by: kim <grufwub@gmail.com>
* cleanup pruneallmeta, add remote header to pruneremote tests
Signed-off-by: kim <grufwub@gmail.com>
* fix olderthan duration additions
Signed-off-by: kim <grufwub@gmail.com>
* fix broken test now that test model header changed
Signed-off-by: kim <grufwub@gmail.com>
* instead use new remote test account for new header model
Signed-off-by: kim <grufwub@gmail.com>
* use newer generated ULID for remote_account_3 to ensure it is sorted last
Signed-off-by: kim <grufwub@gmail.com>
* reorganize serialized keys to match expected test account model order
Signed-off-by: kim <grufwub@gmail.com>
Signed-off-by: kim <grufwub@gmail.com>
* split emoji into local and remote, allow looking up remote emoji by toot url
* optimize some/all filtering
* fix local emoji routes
* implement copy action
* shortcode validation, don't wipe form on error
* copy & disable PATCH
* remove local toot acceptance for testing
* unused import
* parse emoji from account and status, get web_url from status uri
* fix url parse
* submit button loading info
* actually send category
* code cleanup, distinguish between account and status responses
* use loading icons
* fix loading icon on federation page
* require Loading element
* remove unused require
* query explanation, small accessibility tweaks
* [docs] Serve static assets with nginx
This explains how to use nginx to serve static assets and offload GTS
from that responsibility. It also shows how to have nginx add caching
headers to indicate to clients how long they may cache an asset.
* [docs] Move additional nginx config to advanced
This moves a bunch of additional nginx configuration into the Advanced
page instead. It declutters the nginx configuration page.
* update config generator to support nested structs, add cache configuration options
* update envparsing test
* add cache configuration to config parse tests
* set cache configuration in testrig
* move caches to sub-cache "gts" namespace, update envparsing, add cache config docs to example config
Signed-off-by: kim <grufwub@gmail.com>
This explains how nginx can be used to cache webfinger responses and
potentially serve stale responses in case GTS is down. This can be
useful to do in order to ensure webfinger keeps working even if you're
doing some maintenance.
* [chore] Move ShortcodeDomain to its own little util func
* [feature] Add RefetchEmojis function to media manager
* [feature] Expose admin media refresh via admin API
* update following review feedback
- change/fix log levels
- make sure not to try to refetch local emojis
- small style refactoring + comments
* log on emoji refetch start
Signed-off-by: kim <grufwub@gmail.com>
Co-authored-by: kim <grufwub@gmail.com>
When you have multiple templates, they can only be selected by the API, so you can't just open a pull request via the web UI and have the template presented to you. This should fix that by having just one pull request template.
* [docs] Update contributing.md
- Add Pull Request process and guidelines.
- Add feature/bug issue process.
- Rearrange some sections for clarity.
- Add overview of package structure.
* [docs] Add build from source links
* [chore] add pull request templates
These link to the new CONTRIBUTING.md document, and include a checklist to validate that contributors have read the guidelines.
* [docs] Put existing stub CoC in separate doc
* update web related stuff in CONTRIBUTING.md
Co-authored-by: f0x <f0x@cthu.lu>
* Implement Bookmarks
* Update based on review comments
* Update swagger doc
* Fix argument passing to status.Bookmark
* Update changed test
* Updates based on latest PR review
* move caches to a separate State{} structure
Signed-off-by: kim <grufwub@gmail.com>
* fix call to log.Panic not using formatted call
Signed-off-by: kim <grufwub@gmail.com>
* move caches to use interfaces, to make switchouts easier in future
Signed-off-by: kim <grufwub@gmail.com>
* fix rebase issue
Signed-off-by: kim <grufwub@gmail.com>
* improve code comment
Signed-off-by: kim <grufwub@gmail.com>
* fix further issues after rebase
Signed-off-by: kim <grufwub@gmail.com>
* heh
Signed-off-by: kim <grufwub@gmail.com>
* add missing license text
Signed-off-by: kim <grufwub@gmail.com>
Signed-off-by: kim <grufwub@gmail.com>
* [feature] overhaul the oidc system
this allows for more flexible username handling and prevents account
takeover using old email addresses
* [feature] add migration path for old OIDC users
* [feature] nicer error reporting for users
* [docs] document the new OIDC flow
* [fix] return early on oidc error
* [docs]: add comments on the finalization logic
* remove filesystem logging directives from example systemd unit config
* [docs] Update docs to reflect new systemd config
Co-authored-by: tsmethurst <tobi.smethurst@protonmail.com>
Implements #864 and should speed up s3 based installations by a lot.
With more static urls, we can then also implement #1026 for even
better performance when used in conjunction with CDNs
In the previous changes that expanded the IPv4 and IPv6 deny lists based
on the IANA registries we inadvertently added a number of duplicates.
This is unnecessary as they're already caught by larger prefixes and
means there's less entries to scan.
This change removes all prefixes that are subnets of other prefixes.
* ap: add support for PKCS1 "RSA PUBLIC KEY" pem block type
Signed-off-by: Sigrid Solveig Haflínudóttir <sigrid@ftrv.se>
* ap: report no PEM data or unknown pem block type
Signed-off-by: Sigrid Solveig Haflínudóttir <sigrid@ftrv.se>
Signed-off-by: Sigrid Solveig Haflínudóttir <sigrid@ftrv.se>
* Enable the 'admonitions' Markdown extension for Mkdocs.
The admonitions extension to Python-Markdown allows you to include
rST-style "admonitions" to Markdown documents, for instance,
!!! note
Here's an important note to keep in mind!
In general, the current documentation uses bold text to try to achieve
the same effect, which is a bit harder to notice and makes it difficult
to differentiate between "here's something useful to know" versus "here
there be dragons".
* Add AppArmor profile and documentation for LSM-related sandboxing
This commit adds an AppArmor profile for gotosocial in
examples/apparmor/gotosocial. This will (hopefully) serve as a helpful
security mitigation for people are planning on deploying GTS on a
Debian-family Linux distribution.
I've also updates the documentation to include some information about
deploying GTS with either AppArmor or SELinux (moving the documentation
for the former out of the "binary installation guide" docs).
* only return error for emoji fetch if NOT errnoentries
Signed-off-by: kim <grufwub@gmail.com>
* reformat gts->api model slice conversion to standard error behaviours and reduce code reuse
Signed-off-by: kim <grufwub@gmail.com>
Signed-off-by: kim <grufwub@gmail.com>
