Commit graph

108 commits

Author SHA1 Message Date
kim b85a9983d0
[bugfix] fix emoji recaching operations (#3167)
* add test for emoji update image

* update emoji recache to set the instance account id

* don't refresh emoji if only not cached. in that case literally just recache

* code comment

* rename + move a few things

* add some more code comments, and rename some functions to make logic a bit clearer

* remove unnecessary nil check (the value can be nil)

* comment wording

* remove test data output

* handle the case of caching an emoji which has been refreshed then uncached

* allow overwriting on testrig storage as we do now on regular storage

* fix emoji category ID not getting updated

---------

Co-authored-by: tobi <tobi.smethurst@protonmail.com>
2024-08-03 19:05:38 +02:00
tobi 7b5917d6ae
[feature] Allow import of following and blocks via CSV (#3150)
* [feature] Import follows + blocks via settings panel

* test import follows
2024-08-02 12:41:46 +01:00
Vyr Cossont fd837776e2
[feature] Implement Mastodon-compatible roles (#3136)
* Implement Mastodon-compatible roles

- `Account.role` should only be available through verify_credentials for checking current user's permissions
- `Account.role` now carries a Mastodon-compatible permissions bitmap and a marker for whether it should be shown to the public
- `Account.roles` added for *public* display roles (undocumented but stable since Mastodon 4.1)
- Web template now uses only public display roles (no user-visible change here, we already special-cased the `user` role)

* Handle verify_credentials case for default role

* Update JSON exact-match tests

* Address review comments

* Add blocks bit to admin permissions bitmap
2024-07-31 09:26:09 -07:00
tobi c9b6220fef
[chore] Add interaction filter to complement existing visibility filter (#3111)
* [chore] Add interaction filter to complement existing visibility filter

* pass in ptr to visibility and interaction filters to Processor{} to ensure shared

* use int constants for for match type, cache db calls in filterctx

* function name typo 😇

---------

Co-authored-by: kim <grufwub@gmail.com>
2024-07-24 12:27:42 +01:00
tobi ad700c5029
[chore/frontend] Use smaller webp images for logo and default avatars (#3119) 2024-07-20 14:02:22 +01:00
kim 50c9b5498b
[feature] use webp for thumbnails (#3116)
* update to use webp for thumbnails

* bump webp quality up to 40% from 12% (it's a bit different to jpeg quality setting)

* update to use yuva colorspace, and use thumbnail=n=10 to select frame

* fix missing comma in ffmpeg args

* add links to appropriate ffmpeg docs

* update tests

* add file size tests for thumbnails

---------

Co-authored-by: tobi <tobi.smethurst@protonmail.com>
2024-07-19 17:28:43 +02:00
tobi 0aadc2db2a
[feature] Allow users to set default interaction policies per status visibility (#3108)
* [feature] Allow users to set default interaction policies

* use vars for default policies

* avoid some code repetition

* unfuck form binding

* avoid bonkers loop

* beep boop

* put policyValsToAPIPolicyVals in separate function

* don't bother with slices.Grow

* oops
2024-07-17 16:46:52 +02:00
tobi 9efb11d848
[feature/frontend] Add player for audio files; use thumbnail for poster (#3099)
* [feature/frontend] Audio player for audio media types

* use video preview images for previews instead of video itself

* don't preload

* update tests for new zork status

* collapse media gallery into single row when small
2024-07-15 10:47:57 +01:00
kim cde2fb6244
[feature] support processing of (many) more media types (#3090)
* initial work replacing our media decoding / encoding pipeline with ffprobe + ffmpeg

* specify the video codec to use when generating static image from emoji

* update go-storage library (fixes incompatibility after updating go-iotools)

* maintain image aspect ratio when generating a thumbnail for it

* update readme to show go-ffmpreg

* fix a bunch of media tests, move filesize checking to callers of media manager for more flexibility

* remove extra debug from error message

* fix up incorrect function signatures

* update PutFile to just use regular file copy, as changes are file is on separate partition

* fix remaining tests, remove some unneeded tests now we're working with ffmpeg/ffprobe

* update more tests, add more code comments

* add utilities to generate processed emoji / media outputs

* fix remaining tests

* add test for opus media file, add license header to utility cmds

* limit the number of concurrently available ffmpeg / ffprobe instances

* reduce number of instances

* further reduce number of instances

* fix envparsing test with configuration variables

* update docs and configuration with new media-{local,remote}-max-size variables
2024-07-12 09:39:47 +00:00
tobi d70f4e166d
[feature/frontend] Allow setting alt-text for avatar + header (#3086) 2024-07-08 15:47:03 +02:00
kim 21bb324156
[chore] media and emoji refactoring (#3000)
* start updating media manager interface ready for storing attachments / emoji right away

* store emoji and media as uncached immediately, then (re-)cache on Processing{}.Load()

* remove now unused media workers

* fix tests and issues

* fix another test!

* fix emoji activitypub uri setting behaviour, fix remainder of test compilation issues

* fix more tests

* fix (most of) remaining tests, add debouncing to repeatedly failing media / emojis

* whoops, rebase issue

* remove kim's whacky experiments

* do some reshuffling, ensure emoji uri gets set

* ensure marked as not cached on cleanup

* tweaks to media / emoji processing to handle context canceled better

* ensure newly fetched emojis actually get set in returned slice

* use different varnames to be a bit more obvious

* move emoji refresh rate limiting to dereferencer

* add exported dereferencer functions for remote media, use these for recaching in processor

* add check for nil attachment in updateAttachment()

* remove unused emoji and media fields + columns

* see previous commit

* fix old migrations expecting image_updated_at to exists (from copies of old models)

* remove freshness checking code (seems to be broken...)

* fix error arg causing nil ptr exception

* finish documentating functions with comments, slight tweaks to media / emoji deref error logic

* remove some extra unneeded boolean checking

* finish writing documentation (code comments) for exported media manager methods

* undo changes to migration snapshot gtsmodels, updated failing migration to have its own snapshot

* move doesColumnExist() to util.go in migrations package
2024-06-26 16:01:16 +01:00
tobi d2b3d37724
[feature/frontend] Reports frontend v2 (#3022)
* use apiutil + paging in admin processor+handlers

* we're making it happen

* fix little whoopsie

* styling for report list

* don't youuuu forget about meee don't don't don't don't

* last bits

* sanitize content before showing in report statuses

* update report docs
2024-06-18 18:18:00 +02:00
tobi bcda048eab
[feature] Self-serve email change for users (#2957)
* [feature] Email change

* frontend stuff for changing email

* docs

* tests etc

* differentiate more clearly between local user+account and account

* populate user
2024-06-06 14:43:25 +01:00
tobi f9a4a6120d
[feature] Debug admin endpoint to clear caches (#2940)
* [feature] Debug admin endpoint to clear caches

* go fmt
2024-05-29 12:56:17 +02:00
tobi a276b1ca06
[feature/frontend] Let admins send test email to validate SMTP config (#2934)
* [feature/frontend] Let admins send test email to validate SMTP config

* wee
2024-05-27 17:03:54 +00:00
tobi 6171dcbe51
[feature] Add HTTP header permission section to frontend (#2893)
* [feature] Add HTTP header filter section to frontend

* tweak naming a bit
2024-05-05 11:47:22 +00:00
tobi 725a21b027
[feature] Page through accounts as moderator (#2881)
* [feature] Page through accounts as moderator

* aaaaa

* use COLLATE "C" for Postgres to ensure same ordering as SQLite

* fix typo, test paging up

* don't show moderation / info for our instance acct
2024-05-01 14:11:22 +01:00
tobi 89e0cfd874
[feature] Admin accounts endpoints; approve/reject sign-ups (#2826)
* update settings panels, add pending overview + approve/deny functions

* add admin accounts get, approve, reject

* send approved/rejected emails

* use signup URL

* docs!

* email

* swagger

* web linting

* fix email tests

* wee lil fixerinos

* use new paging logic for GetAccounts() series of admin endpoints, small changes to query building

* shuffle useAccountIDIn check *before* adding to query

* fix parse from toot react error

* use `netip.Addr`

* put valid slices in globals

* optimistic updates for account state

---------

Co-authored-by: kim <grufwub@gmail.com>
2024-04-13 13:25:10 +02:00
tobi 9fb8a78f91
[feature] New user sign-up via web page (#2796)
* [feature] User sign-up form and admin notifs

* add chosen + filtered languages to migration

* remove stray comment

* chosen languages schmosen schmanguages

* proper error on local account missing
2024-04-11 11:45:53 +02:00
tobi f05874be30
[feature] Option to hide followers/following (#2788) 2024-04-02 10:42:24 +01:00
tobi ab2d063fcb
[feature] Process outgoing Move from clientAPI (#2750)
* prevent moved accounts from taking create-type actions

* update move logic

* federate move out

* indicate on web profile when an account has moved

* [docs] Add migration docs section

* lock while checking + setting move state

* use redirectFollowers func for clientAPI as well

* comment typo

* linter? i barely know 'er!

* Update internal/uris/uri.go

Co-authored-by: Daenney <daenney@users.noreply.github.com>

* add a couple tests for move

* fix little mistake exposed by tests (thanks tests)

* ensure Move marked as successful

* attach shared util funcs to struct

* lock whole account when doing move

* move moving check to after error check

* replace repeated text with error func

* linterrrrrr!!!!

* catch self follow case

---------

Co-authored-by: Daenney <daenney@users.noreply.github.com>
2024-03-13 13:53:29 +01:00
Vyr Cossont 13b9fd5f92
[bugfix]: Add missing Link headers in Swagger spec (#2751)
* Adds Link headers to all endpoints that use it for paging
* Fixes the return type for listing what accounts someone follows
2024-03-13 12:23:26 +01:00
Vyr Cossont fc3741365c
[bugfix] Fix Swagger spec and add test script (#2698)
* Add Swagger spec test script

* Fix Swagger spec errors not related to statuses with polls

* Add API tests that post a status with a poll

* Fix creating a status with a poll from form params

* Fix Swagger spec errors related to statuses with polls (this is the last error)

* Fix Swagger spec warnings not related to unused definitions

* Suppress a duplicate list update params definition that was somehow causing wrong param names

* Add Swagger test to CI

- updates Drone config
- vendorizes go-swagger
- fixes a file extension issue that caused the test script to generate JSON instead of YAML with the vendorized version

* Put `Sample: ` on its own line everywhere

* Remove unused id param from emojiCategoriesGet

* Add 5 more pairs of profile fields to account update API Swagger

* Remove Swagger prefix from dummy fields

It makes the generated code look weird

* Manually annotate params for statusCreate operation

* Fix all remaining Swagger spec warnings

- Change some models into operation parameters
- Ignore models that already correspond to manually documented operation parameters but can't be trivially changed (those with file fields)

* Documented that creating a status with scheduled_at isn't implemented yet

* sign drone.yml

* Fix filter API Swagger errors

* fixup! Fix filter API Swagger errors

---------

Co-authored-by: tobi <tobi.smethurst@protonmail.com>
2024-03-06 18:05:45 +01:00
tobi 9cadc764b3
[feature] Add experimental instance-federation-spam-filter option (#2685)
* [chore] Move `visibility` to `filter/visibility`

* [feature] Add experimental instance-federation-spam-filter option
2024-02-27 12:22:05 +00:00
tobi aa8bbe6ad2
[bugfix] Fix Postgres emoji delete, emoji category change (#2570)
* [bugfix] Fix Postgres emoji delete, emoji category change

* revert trace logging

* caching issue

* update tests
2024-01-29 14:57:22 +00:00
kim 97a1fd9a29
[bugfix] 😇 (#2476) 2023-12-18 16:40:29 +00:00
kim 8ebb7775a3
[feature] request blocking by http headers (#2409) 2023-12-18 14:18:25 +00:00
tobi 5eddef6c9b
[feature] Add /api/v1/admin/debug/apurl endpoint (#2359) 2023-11-27 14:02:52 +00:00
kim 74700cc803
[performance] http response encoding / writing improvements (#2374) 2023-11-27 14:00:57 +00:00
tobi 4ee436e98a
[bugfix] process account delete side effects in serial, not in parallel (#2360)
* [bugfix] process account delete side effects in serial, not in parallel

* StartWorkers / StartNoopWorkers for tests

* undo testrig trace logging

* log errors instead of immediately returning
2023-11-14 15:57:25 +01:00
kim deaea100c3
[bugfix] support endless polls, and misskey's' method of inferring expiry in closed polls (#2349) 2023-11-11 10:15:04 +00:00
tobi c7ecab9e6f
[chore/bugfix/horror] Allow expires_in and poll choices to be parsed from strings (#2346) 2023-11-10 16:42:48 +00:00
kim e9e5dc5a40
[feature] add support for polls + receiving federated status edits (#2330) 2023-11-08 14:32:17 +00:00
kim 69ba9a79a1
[chore] de-interface{} the federator and dereferencer structs (#2285)
* de-interface{} the federator and dereferencer structs

* fix broken type signatures
2023-10-23 10:58:13 +01:00
tobi 637f188ebe
[feature] Allow import/export/creation of domain allows via admin panel (#2264)
* it's happening!

* aaa

* fix silly whoopsie

* it's working pa! it's working ma!

* model report parameters

* shuffle some more stuff around

* getting there

* oo hoo

* finish tidying up for now

* aaa

* fix use form submit errors

* peepee poo poo

* aaaaa

* ffff

* they see me typin', they hatin'

* boop

* aaa

* oooo

* typing typing tappa tappa

* almost done typing

* weee

* alright

* push it push it real good doo doo doo doo doo doo

* thingy no worky

* almost done

* mutation modifers not quite right

* hmm

* it works

* view blocks + allows nicely

* it works!

* typia install

* the old linterino

* linter plz
2023-10-17 12:46:06 +02:00
kim c6e00afc7c
[feature] tentatively start adding polls support (#2249) 2023-10-04 13:09:42 +01:00
kim 8f67dd583d
[chore] deinterface the typeutils.Converter and update to use state structure (#2217)
* update typeconverter to use state structure

* deinterface the typeutils.TypeConverter -> typeutils.Converter

* finish copying over old type converter code comments

* fix cherry-pick merge issues, fix tests pointing to old typeutils interface type still
2023-09-23 18:44:11 +02:00
tobi 183eaa5b29
[feature] Implement explicit domain allows + allowlist federation mode (#2200)
* love like winter! wohoah, wohoah

* domain allow side effects

* tests! logging! unallow!

* document federation modes

* linty linterson

* test

* further adventures in documentation

* finish up domain block documentation (i think)

* change wording a wee little bit

* docs, example

* consolidate shared domainPermission code

* call mode once

* fetch federation mode within domain blocked func

* read domain perm import in streaming manner

* don't use pointer to slice for domain perms

* don't bother copying blocks + allows before deleting

* admonish!

* change wording just a scooch

* update docs
2023-09-21 12:12:04 +02:00
tobi 4b594516ec
[feature] Allow admins to expire remote public keys; refetch expired keys on demand (#2183) 2023-09-12 10:43:12 +01:00
tobi 3ed1ca68e5
[feature] Store admin actions in the db, prevent conflicting actions (#2167) 2023-09-04 14:55:17 +01:00
f0x52 92de8fb396
[feature] Instance rules (#2125)
* init instance rules database model, admin api

* expose instance rules in public instance api

* public /api/v1/instance/rules route

* GET ruleById

* createRule route

* createRule auth check

* updateRule

* deleteRule

* list rules on about page

* ruleGet auth

* add about page ids for anchors

* process and store adding violated rules to reports

* admin api models for instance rules

* instance rule edit frontend

* change rule inputs to textareas

* database fixes after rebase (#2124)

* remove unused imports

* fix db migration column name

* fix tests

* fix more tests

* fix postgres error with wrongly used Ident

* add some tests, fiddle with rule model a bit, fix postgres migration

* swagger docs

---------

Co-authored-by: tsmethurst <tobi.smethurst@protonmail.com>
2023-08-19 14:33:15 +02:00
Daenney be3718f6e4
[chore] Use generic pointer function (#2080)
This replaces the different $TypePtr functions with a generic
implementation.
2023-08-07 18:38:11 +01:00
tobi e8a20f587c
[bugfix] Rework MultiError to wrap + unwrap errors properly (#2057)
* rework multierror a bit

* test multierror
2023-08-02 17:21:46 +02:00
tobi e70bf8a6c8
[chore/bugfix] Domain block tidying up, Implement first pass of 207 Multi-Status (#1886)
* [chore/refactor] update domain block processing

* expose domain block import errors a lil better

* move/remove unused query keys
2023-07-07 11:34:12 +02:00
Julian-Samuel Gebühr 1f39275c0f
[bugfix] Allow lowercase emoji shortcode in frontend (#1851)
* Replace pinafore with semaphore

* Typo

* Allow lowercase emoji shortcode in frontend

* Fix failing test (corrected expected outcome)
2023-06-02 17:42:14 +02:00
kim 5faeb4de20
[chore] tidy up media manager, add calling func to errors, build-script improvements (#1835)
* media manager tidy-up: de-interface and remove unused PostDataFunc

Signed-off-by: kim <grufwub@gmail.com>

* remove last traces of media.Manager being an interface

Signed-off-by: kim <grufwub@gmail.com>

* update error to provide caller, allow tuneable via build tags

Signed-off-by: kim <grufwub@gmail.com>

* remove kim-specific build script changes

Signed-off-by: kim <grufwub@gmail.com>

* fix merge conflicts

Signed-off-by: kim <grufwub@gmail.com>

* update build-script to support externally setting build variables

Signed-off-by: kim <grufwub@gmail.com>

---------

Signed-off-by: kim <grufwub@gmail.com>
2023-05-28 14:08:35 +02:00
tobi f5c004d67d
[feature] Add List functionality (#1802)
* start working on lists

* further list work

* test list db functions nicely

* more work on lists

* peepoopeepoo

* poke

* start list timeline func

* we're getting there lads

* couldn't be me working on stuff... could it?

* hook up handlers

* fiddling

* weeee

* woah

* screaming, pissing

* fix streaming being a whiny baby

* lint, small test fix, swagger

* tidying up, testing

* fucked! by the linter

* move timelines to state like a boss

* add timeline start to tests using state

* invalidate lists
2023-05-25 10:37:38 +02:00
f0x52 89dcbd5a20
[frontend] Basic user moderation actions (#1728)
* remove info banner

* update swagger definition for AccountAction

* basic user view, suspend action

* clean up suspended user display

* basic user searching

* rename User -> Account for clarity

* refactor error boundary component to give better info

* appease the linter
2023-05-13 12:17:22 +02:00
tobi 0e29f1f5bb
[feature] Enable federation in/out of profile PropertyValue fields (#1722)
Co-authored-by: kim <grufwub@gmail.com>
Co-authored-by: kim <89579420+NyaaaWhatsUpDoc@users.noreply.github.com>
2023-05-09 11:16:10 +01:00
tobi d9bbcc60a6
[bugfix] Fix report serialization errors caused by user delete (#1659)
* [bugfix] Fix report serialization errors caused by user delete

* fix tests
2023-03-31 15:01:29 +02:00