Fixes an issue where async processing was not completing correctly.
In particular this applies to side effects of domain blocks: while the domain block was being entered and enforced correctly, side effects like deleting accounts and updating the instance entry for the blocked instance were not. This fixes that :)
* update dependencies, bump Go version to 1.19
* bump test image Go version
* update golangci-lint
* update gotosocial-drone-build
* sign
* linting, go fmt
* update swagger docs
* update swagger docs
* whitespace
* update contributing.md
* fuckin whoopsie doopsie
* linterino, linteroni
* fix followrequest test not starting processor
* fix other api/client tests not starting processor
* fix remaining tests where processor not started
* bump go-runners version
* don't check last-webfingered-at, processor may have updated this
* update swagger command
* update bun to latest version
* fix embed to work the same as before with new bun
Signed-off-by: kim <grufwub@gmail.com>
Co-authored-by: tsmethurst <tobi.smethurst@protonmail.com>
* Start adding account emoji
* get emojis serialized + deserialized nicely
* update tests
* set / retrieve emojis on accounts
* show account emojis in web view
* fetch emojis from db based on ids
* fix typo in test
* lint
* fix pg migration
* update tests
* update emoji checking logic
* update comment
* clarify comments + add some spacing
* tidy up loops a lil (thanks kim)
* update thread iterators to not use recursion, rewrote both
Signed-off-by: kim <grufwub@gmail.com>
* fix endless descendant deref, don't error if fetching existing status
Signed-off-by: kim <grufwub@gmail.com>
* don't refetch remote ancestor statuses, improve descendant iter commenting
Signed-off-by: kim <grufwub@gmail.com>
* move collection page next logic so we capture first page of entities
Signed-off-by: kim <grufwub@gmail.com>
* improve log format argument quoting
Signed-off-by: kim <grufwub@gmail.com>
* improve code commenting of collection paging
Signed-off-by: kim <grufwub@gmail.com>
* only dereference announce's originating status if _not_ local. update DereferenceThread() signature. cleanup searchStatusByURI()
Signed-off-by: kim <grufwub@gmail.com>
Signed-off-by: kim <grufwub@gmail.com>
* update Activity
* add instance-deliver-to-shared-inboxes setting
* update activity version again
* add SharedInboxURI field to accounts
* serdes for endpoints/sharedInbox
* deliver to sharedInbox if one is available
* update tests
* only assign shared inbox if shared domain
* look for shared inbox if currently nil
* go fmt
* finger to get params.RemoteAccountID if necessary
* make comments clearer
* compare dns more consistently
* add func for deleting status from db + cache
* move deletes entirely back to processor
and also only do a delete if the requesting account owns the item being deleted
* tidy up unboost processing
* delete status more efficiently
* fix wrong account id on remote test attachments
* fix federator test
* add custom css account property + db func to fetch
* allow account to get/set custom css
* serve custom css for an account
* go fmt
* use monospace for customcss, add link
* add custom css to account cache
* fix broken field
* add custom css docs to user guide
* add `accounts-allow-custom-css` config flag
* add allow custom css to /api/v1/instance response
* only show/set custom css if allowed to do so
* only set/serve custom account css if enabled
* update swagger docs
* chain promise
* make bool a bit clearer
* use cache for GetAccountCustomCSSByUsername
* start implementing remote emoji fetcher
* update status where pk
* aaa
* tidy up a little
* check size limits for emojis
* thank you linter, i love you <3
* update swagger docs
* add emoji dereference test
* make emoji max sizes configurable
* normalize db.ErrAlreadyExists
* migrate emojis
* add get emoji to s2s (federation) API
* add new emoji db + cache functions
* add shortcodeDomain lookup for emojis
* check existing emojis w/cache, not w/constraints
* go fmt
* add putEmoji func
* use new db emoji funcs instead of where
* remove emojistringstotags func
* add unique constraint back in
* fix up broken migration
* update index
* validate web-asset-base-dir
* move default icons into converter
* always ensure avatar + header on api accounts
* update tests
* add default header
* don't return error from web module creation anymore
* tidy a bit
* use pngs for default avatars rather than svgs
* fetch creation and fetching domain blocks from db
Signed-off-by: kim <grufwub@gmail.com>
* add separate domainblock cache type, handle removing block from cache on delete
Signed-off-by: kim <grufwub@gmail.com>
* fix sentinel nil values being passed into cache
Signed-off-by: kim <grufwub@gmail.com>
Signed-off-by: kim <grufwub@gmail.com>
* use 'test' value for testrig storage backend
* update test dependency
* add WaitFor func in testrig
* use WaitFor function instead of time.Sleep
* tidy up tests
* make SentMessages a sync.map
* go fmt
* Fix some bugs when viewing a user's posts: include their self-replies (threads) even when excludeReplies is set, and use in_reply_to_uri instead of in_reply_to_id to filter out replies
* Assign values to InReplyToURI when creating statuses. Add index and update old statuses with a migration
* upstep bun and sqlite versions
* allow specific columns to be updated in the db
* only update necessary columns for user
* bit tidier
* only update necessary fields of media_attachment
* only update relevant instance fields
* update tests
* update only specific account columns
* use bool pointers on gtsmodels
includes attachment, status, account, user
* update columns more selectively
* test all default fields on new account insert
* updating remaining bools on gtsmodels
* initialize pointer fields when extracting AP emoji
* copy bools properly
* add copyBoolPtr convenience function + test it
* initialize false bool ptrs a bit more neatly
* start work on user panel
* parse source first before checking if empty form
* newline
* set avi + header nicely
* add posts settings
* render signin a bit nicer on mobile
* return OK json on successful change
* return unauthorized on bad password
* clarify message on insecure password
* make login a bit prettier
* add alt text + border round image previews
* add logout button
* add password change
* styling updates
* redirect /auth/edit to /user
* update tests
* fix validation tests
* better labels, link to more info
* make submit button generic component
* move submit button inside forms
* add autocomplete labels to password fields
* fix indentation (thx eslint)
* update eslintrc
* eslint: no-unescaped-entities
* initial deduplication between user and admin panel
* add default status/post format setting
* user panel styling for inputs
* update user panel styling, include normalize css
* add placeholder text
* input padding
Co-authored-by: f0x <f0x@cthu.lu>
* add post_format to acct & use it when making post
* update swagger docs
* add status_format updating to frontend
* fix up tests
* post_format => status_format
* add status_format to account validation
* first commit
Signed-off-by: kim <grufwub@gmail.com>
* replace logging with our own log library
Signed-off-by: kim <grufwub@gmail.com>
* fix imports
Signed-off-by: kim <grufwub@gmail.com>
* fix log imports
Signed-off-by: kim <grufwub@gmail.com>
* add license text
Signed-off-by: kim <grufwub@gmail.com>
* fix package import cycle between config and log package
Signed-off-by: kim <grufwub@gmail.com>
* fix empty kv.Fields{} being passed to WithFields()
Signed-off-by: kim <grufwub@gmail.com>
* fix uses of log.WithFields() with whitespace issues and empty slices
Signed-off-by: kim <grufwub@gmail.com>
* *linter related grumbling*
Signed-off-by: kim <grufwub@gmail.com>
* gofmt the codebase! also fix more log.WithFields() formatting issues
Signed-off-by: kim <grufwub@gmail.com>
* update testrig code to match new changes
Signed-off-by: kim <grufwub@gmail.com>
* fix error wrapping in non fmt.Errorf function
Signed-off-by: kim <grufwub@gmail.com>
* add benchmarking of log.Caller() vs non-cached
Signed-off-by: kim <grufwub@gmail.com>
* fix syslog tests, add standard build tags to test runner to ensure consistency
Signed-off-by: kim <grufwub@gmail.com>
* make syslog tests more robust
Signed-off-by: kim <grufwub@gmail.com>
* fix caller depth arithmatic (is that how you spell it?)
Signed-off-by: kim <grufwub@gmail.com>
* update to use unkeyed fields in kv.Field{} instances
Signed-off-by: kim <grufwub@gmail.com>
* update go-kv library
Signed-off-by: kim <grufwub@gmail.com>
* update libraries list
Signed-off-by: kim <grufwub@gmail.com>
* fuck you linter get nerfed
Signed-off-by: kim <grufwub@gmail.com>
Co-authored-by: tobi <31960611+tsmethurst@users.noreply.github.com>
* add GetAccountWebStatuses to db
* add WebStatusesGet func to processor
* don't add limit to next/prev links if 0
* take query params for next/prev statuses
* add separate next + prev links for convenience
* show 'nothing here' message if no statuses exist
* add back / next links to profiles
* allow paging down only
* go fmt ./...
* 'recent public toots' -> 'latest public toots'
* add miekg/dns dependency
* set/validate accountDomain
* move finger to dereferencer
* totally break GetRemoteAccount
* start reworking finger func a bit
* start reworking getRemoteAccount a bit
* move mention parts to namestring
* rework webfingerget
* use util function to extract webfinger parts
* use accountDomain
* rework finger again, final form
* just a real nasty commit, the worst
* remove refresh from account
* use new ASRepToAccount signature
* fix incorrect debug call
* fix for new getRemoteAccount
* rework GetRemoteAccount
* start updating tests to remove repetition
* break a lot of tests
Move shared test logic into the testrig,
rather than having it scattered all over
the place. This allows us to just mock
the transport controller once, and have
all tests use it (unless they need not to
for some other reason).
* fix up tests to use main mock httpclient
* webfinger only if necessary
* cheeky linting with the lads
* update mentionName regex
recognize instance accounts
* don't finger instance accounts
* test webfinger part extraction
* increase default worker count to 4 per cpu
* don't repeat regex parsing
* final search for discovered accountDomain
* be more permissive in namestring lookup
* add more extraction tests
* simplify GetParseMentionFunc
* skip long search if local account
* fix broken test
* consolidate to all use same caching libraries
Signed-off-by: kim <grufwub@gmail.com>
* perform more caching in the database layer
Signed-off-by: kim <grufwub@gmail.com>
* remove ASNote cache
Signed-off-by: kim <grufwub@gmail.com>
* update cache library, improve db tracing hooks
Signed-off-by: kim <grufwub@gmail.com>
* return ErrNoEntries if no account status IDs found, small formatting changes
Signed-off-by: kim <grufwub@gmail.com>
* fix tests, thanks tobi!
Signed-off-by: kim <grufwub@gmail.com>
Co-authored-by: tsmethurst <tobi.smethurst@protonmail.com>
* feat: vendor minio client
* feat: introduce storage package with s3 support
* feat: serve s3 files directly
this saves a lot of bandwith as the files are fetched from the object
store directly
* fix: use explicit local storage in tests
* feat: integrate s3 storage with the main server
* fix: add s3 config to cli tests
* docs: explicitly set values in example config
also adds license header to the storage package
* fix: use better http status code on s3 redirect
HTTP 302 Found is the best fit, as it signifies that the resource
requested was found but not under its presumed URL
307/TemporaryRedirect would mean that this resource is usually located
here, not in this case
303/SeeOther indicates that the redirection does not link to the
requested resource but to another page
* refactor: use context in storage driver interface
* add localUnattached db function
* add parseOlderThan util function
* add pruneunusedlocalattachments to media manager
* add unusedlocal pruning to schedule + admin call
* set number of days to keep as a const
* fix test
* add missing license headers
* start adding instance peers get
* rename domainblock.go
* embed domain in domainblock so it can be reused
* update swagger docs
* add test instances to db
* update tests
* add/update instancepeersget
* update domain model
* add getinstancepeers to db
* instance-expose-peers, instance-expose-suspended
* add auth checks for both current filters
* attach endpoint to router
* include public comment
* obfuscate domain if required
* go mod tidy
* update swagger docs
* remove unnecessary comment
* return 'flat' peerlist if no query params provided
* move finger to dereferencer
* totally break GetRemoteAccount
* start reworking finger func a bit
* start reworking getRemoteAccount a bit
* move mention parts to namestring
* rework webfingerget
* use util function to extract webfinger parts
* use accountDomain
* rework finger again, final form
* just a real nasty commit, the worst
* remove refresh from account
* use new ASRepToAccount signature
* fix incorrect debug call
* fix for new getRemoteAccount
* rework GetRemoteAccount
* start updating tests to remove repetition
* break a lot of tests
Move shared test logic into the testrig,
rather than having it scattered all over
the place. This allows us to just mock
the transport controller once, and have
all tests use it (unless they need not to
for some other reason).
* fix up tests to use main mock httpclient
* webfinger only if necessary
* cheeky linting with the lads
* update mentionName regex
recognize instance accounts
* don't finger instance accounts
* test webfinger part extraction
* increase default worker count to 4 per cpu
* don't repeat regex parsing
* final search for discovered accountDomain
* be more permissive in namestring lookup
* add more extraction tests
* simplify GetParseMentionFunc
* skip long search if local account
* fix broken test
* start fiddling about with oauth server
* start returning more helpful errors from oauth
* test helpful(ish) token errors
* add missing license header
* update templates
* start reworking api error handling
* update template
* return AP status at web endpoint if negotiated
* start making api error handling much more consistent
* update account endpoints to new error handling
* use new api error handling in admin endpoints
* go fmt ./...
* use api error logic in app
* use generic error handling in auth
* don't export generic error handler
* don't defer clearing session
* user nicer error handling on oidc callback handler
* tidy up the sign in handler
* tidy up the token handler
* use nicer error handling in blocksget
* auth emojis endpoint
* fix up remaining api endpoints
* fix whoopsie during login flow
* regenerate swagger docs
* change http error logging to debug
* test link headers
* page get account statuses properly
* page get notifications
* add util func for packaging timeline responses
* return timelined stuff from accountstatusesget
* rename timeline response
* use new convenience function
* go fmt
* move config flag names and usage to config package, rewrite config package to use global Configuration{} struct
Signed-off-by: kim <grufwub@gmail.com>
* improved code comment
Signed-off-by: kim <grufwub@gmail.com>
* linter
Signed-off-by: kim <grufwub@gmail.com>
* fix unmarshaling
Signed-off-by: kim <grufwub@gmail.com>
* remove kim's custom go compiler changes
Signed-off-by: kim <grufwub@gmail.com>
* generate setter and flag-name functions, implement these in codebase
Signed-off-by: kim <grufwub@gmail.com>
* update deps
Signed-off-by: kim <grufwub@gmail.com>
* small change
Signed-off-by: kim <grufwub@gmail.com>
* appease the linter...
Signed-off-by: kim <grufwub@gmail.com>
* move configuration into ConfigState structure, ensure reloading to/from viper settings to keep in sync
Signed-off-by: kim <grufwub@gmail.com>
* lint
Signed-off-by: kim <grufwub@gmail.com>
* update code comments
Signed-off-by: kim <grufwub@gmail.com>
* fix merge issue
Signed-off-by: kim <grufwub@gmail.com>
* fix merge issue
Signed-off-by: kim <grufwub@gmail.com>
* improved version string (removes time + go version)
Signed-off-by: kim <grufwub@gmail.com>
* fix version string build to pass test script + consolidate logic in func
Signed-off-by: kim <grufwub@gmail.com>
* add license text, update config.Defaults comment
Signed-off-by: kim <grufwub@gmail.com>
* add license text to generated config helpers file
Signed-off-by: kim <grufwub@gmail.com>
* defer unlock on config.Set___(), to ensure unlocked on panic
Signed-off-by: kim <grufwub@gmail.com>
* make it more obvious which cmd flags are being attached
Signed-off-by: kim <grufwub@gmail.com>
* don't omitempty on description
* don't omitempty on any fields
* add ms to timestamp format
* don't omitempty on text_url
* rearrange attachment fields a bit
* just give URL again as attachment text url
* update tests
* fix accidental replace
* move caption sanitization -> sanitize.go
* use sanitizeplaintext rather than removehtml
* rename sanitizecaption to sanitizeplaintext
* avoid removing html twice from statuses
* unexport remoteHTML
it's no longer used outside the text package so this
makes it less confusing
* test instance PATCH
* add some announces to test models
* start on announce test logic
* test federatingDB.Announce
* change signature of GetRemoteStatus
* remove 'refresh' logic and replace it with refetch
* go fmt
* remove timeline manager from processor test
* make zork created at determinate
* test get account statuses
* test get + serialize zork
* make account keys determinate
* make admin accountCreate time determinate
* test account to as
* init test config before test log
* test status to frontend
* remove daft Within check
* hack around a bit
* use index of slice
