* move caption sanitization -> sanitize.go
* use sanitizeplaintext rather than removehtml
* rename sanitizecaption to sanitizeplaintext
* avoid removing html twice from statuses
* unexport remoteHTML
it's no longer used outside the text package so this
makes it less confusing
* test instance PATCH
* add some announces to test models
* start on announce test logic
* test federatingDB.Announce
* change signature of GetRemoteStatus
* remove 'refresh' logic and replace it with refetch
* go fmt
* remove timeline manager from processor test
* make zork created at determinate
* test get account statuses
* test get + serialize zork
* make account keys determinate
* make admin accountCreate time determinate
* test account to as
* init test config before test log
* test status to frontend
* remove daft Within check
* hack around a bit
* use index of slice
* tidy up context keys, add otherInvolvedIRIs
* add ReplyToable interface
* skip block check if we own the requesting domain
* add block check for other involved IRIs
* use cacheable status fetch
* remove unused ContextActivity
* remove unused ContextActivity
* add helper for unique URIs
* check through CCs and clean slice
* add GetAccountIDForStatusURI
* add GetAccountIDForAccountURI
* check blocks on involved account
* add statuses to tests
* add some blocked tests
* go fmt
* extract Tos as well as CCs
* test PostInboxRequestBodyHook
* add some more testActivities
* deduplicate involvedAccountIDs
* go fmt
* use cacheable db functions, remove new functions
* implement custom_emojis api endpoint
* add tests for getting custom emoji out of the database and converting to api emoji
* change sort direction of emoji query
* change logging level and initialize array with known length as per kim's suggestions
* add continue to lessen risk of making a malformed struct during conversion from db to api emojis
* recursively check timelineability of parent status
* check following status creator
* add tests for hometimelineability (whew)
* add test with mix of public + unlocked vis
* move status wiping logic to fromcommon.go
* delete reblogs of status when a status is deleted
* add admin boost of zork to test model
* update tests to make them more determinate
* Merge branch 'main' into status_reblog_cleanup
* move status wiping logic to fromcommon.go
* delete reblogs of status when a status is deleted
* add admin boost of zork to test model
* update tests to make them more determinate
* Merge branch 'main' into status_reblog_cleanup
* test status delete via client api
* go fmt
* add optional config validation
* clarify that host and protocol are required
* add validation for host and protocol
* pass prerunArgs as a struct (validate by default)
* rename + tidy up remote pruning
* fix media attachment account join
see https://bun.uptrace.dev/guide/golang-orm.html#table-relationships
* update logging to new function name
* add get avatars and headers to bun
* add pruneallmeta function
* don't set uncached since we're deleting anyway
* fix totalPruned being in wrong place
* test pruning meta
* go fmt ./...
* rename mediaprune
* add meta pruning to routine mediaprune
* tidy up cleanup job scheduling
* rename adminmediaremoteprune
* update mediacleanup to use renamed prune func
* update swagger docs a little bit
* reuse cancel + context
* cache transports in controller by privkey-generated pubkey, add retry logic to transport requests
Signed-off-by: kim <grufwub@gmail.com>
* update code comments, defer mutex unlocks
Signed-off-by: kim <grufwub@gmail.com>
* add count to 'performing request' log message
Signed-off-by: kim <grufwub@gmail.com>
* reduce repeated conversions of same url.URL object
Signed-off-by: kim <grufwub@gmail.com>
* move worker.Worker to concurrency subpackage, add WorkQueue type, limit transport http client use by WorkQueue
Signed-off-by: kim <grufwub@gmail.com>
* fix security advisories regarding max outgoing conns, max rsp body size
- implemented by a new httpclient.Client{} that wraps an underlying
client with a queue to limit connections, and limit reader wrapping
a response body with a configured maximum size
- update pub.HttpClient args passed around to be this new httpclient.Client{}
Signed-off-by: kim <grufwub@gmail.com>
* add httpclient tests, move ip validation to separate package + change mechanism
Signed-off-by: kim <grufwub@gmail.com>
* fix merge conflicts
Signed-off-by: kim <grufwub@gmail.com>
* use singular mutex in transport rather than separate signer mus
Signed-off-by: kim <grufwub@gmail.com>
* improved useragent string
Signed-off-by: kim <grufwub@gmail.com>
* add note regarding missing test
Signed-off-by: kim <grufwub@gmail.com>
* remove useragent field from transport (instead store in controller)
Signed-off-by: kim <grufwub@gmail.com>
* shutup linter
Signed-off-by: kim <grufwub@gmail.com>
* reset other signing headers on each loop iteration
Signed-off-by: kim <grufwub@gmail.com>
* respect request ctx during retry-backoff sleep period
Signed-off-by: kim <grufwub@gmail.com>
* use external pkg with docs explaining performance "hack"
Signed-off-by: kim <grufwub@gmail.com>
* use http package constants instead of string method literals
Signed-off-by: kim <grufwub@gmail.com>
* add license file headers
Signed-off-by: kim <grufwub@gmail.com>
* update code comment to match new func names
Signed-off-by: kim <grufwub@gmail.com>
* updates to user-agent string
Signed-off-by: kim <grufwub@gmail.com>
* update signed testrig models to fit with new transport logic (instead uses separate signer now)
Signed-off-by: kim <grufwub@gmail.com>
* fuck you linter
Signed-off-by: kim <grufwub@gmail.com>
* update LE server to use copy of main HTTP server to maintain server timeouts etc
Signed-off-by: kim <grufwub@gmail.com>
* shutup linter
Signed-off-by: kim <grufwub@gmail.com>
* fix error check of prune to allow missing files
* update go-store library, add test for pruning item with db entry but no file
Signed-off-by: kim <grufwub@gmail.com>
* remove now-unneccessary error check
Signed-off-by: kim <grufwub@gmail.com>
Co-authored-by: kim <grufwub@gmail.com>
* fix existing bio text showing as HTML
- updated replaced mentions to include instance
- strips HTML from account source note in Verify handler
- update text formatter to use buffers for string writes
Signed-off-by: kim <grufwub@gmail.com>
* go away linter
Signed-off-by: kim <grufwub@gmail.com>
* change buf reset location, change html mention tags
Signed-off-by: kim <grufwub@gmail.com>
* reduce FindLinks code complexity
Signed-off-by: kim <grufwub@gmail.com>
* fix HTML to text conversion
Signed-off-by: kim <grufwub@gmail.com>
* Update internal/regexes/regexes.go
Co-authored-by: Mina Galić <mina.galic@puppet.com>
* use improved html2text lib with more options
Signed-off-by: kim <grufwub@gmail.com>
* fix to produce actual plaintext from html
Signed-off-by: kim <grufwub@gmail.com>
* fix span tags instead written as space
Signed-off-by: kim <grufwub@gmail.com>
* performance improvements to regex replacements, fix link replace logic for un-html-ing in the future
Signed-off-by: kim <grufwub@gmail.com>
* fix tag/mention replacements to use input string, fix link replace to not include scheme
Signed-off-by: kim <grufwub@gmail.com>
* use matched input string for link replace href text
Signed-off-by: kim <grufwub@gmail.com>
* remove unused code (to appease linter :sobs:)
Signed-off-by: kim <grufwub@gmail.com>
* improve hashtagFinger regex to be more compliant
Signed-off-by: kim <grufwub@gmail.com>
* update breakReplacer to include both unix and windows line endings
Signed-off-by: kim <grufwub@gmail.com>
* add NoteRaw field to Account to store plaintext account bio, add migration for this, set for sensitive accounts
Signed-off-by: kim <grufwub@gmail.com>
* drop unnecessary code
Signed-off-by: kim <grufwub@gmail.com>
* update text package tests to fix logic changes
Signed-off-by: kim <grufwub@gmail.com>
* add raw note content testing to account update and account verify
Signed-off-by: kim <grufwub@gmail.com>
* remove unused modules
Signed-off-by: kim <grufwub@gmail.com>
* fix emoji regex
Signed-off-by: kim <grufwub@gmail.com>
* fix replacement of hashtags
Signed-off-by: kim <grufwub@gmail.com>
* update code comment
Signed-off-by: kim <grufwub@gmail.com>
Co-authored-by: Mina Galić <mina.galic@puppet.com>
* improved logrus output switching performance
Signed-off-by: kim <grufwub@gmail.com>
* add license to test file
Signed-off-by: kim <grufwub@gmail.com>
* update media manager to use internal/worker package, update worker with better logging
Signed-off-by: kim <grufwub@gmail.com>
* fix Queue() trace log message format operators
Signed-off-by: kim <grufwub@gmail.com>
* update media manager comment to match updated worker implementation
Signed-off-by: kim <grufwub@gmail.com>
* create visibility filter for boostability and allow self-boosting for any visbility but direct messages
* add a followers-only status to local_account_2
* fix typo in comment
* add license header, unwrap errors, be explicit about non-boostable visibility settings to avoid rogue boosting from miscoded clients, use ID compare for checking if self-boosting
* add tests for statusboostable filter
* fix tests that were affected by adding a new status to the test data
* fix the rest of tests affected by adding a status to the textrig data
* remove unnecessary LOWER() db calls
* warn during slow db queries
* use bundb built-in exists function
* add db block test
* update account block query
* add domain block db test
* optimize domain block query
* fix implementing wrong test
* exclude most columns when checking block
* go fmt
* remote more unnecessary use of LOWER()
* tidy up some federation tests
* add missing license to following.go
* give zork some followers, as a treat
* test InboxesForIRI
* Go fmt
* update timeline tests
Only attempt to join the `WebTemplateBaseDir` with the current working
directory if the user has not configured an absolute path to the
template directory.
This changeset also makes a similar change to the testrig, allowing
tests to configure an alternative location for the templates directory.
Fixes#411
Signed-off-by: Terin Stock <terinjokes@gmail.com>
* wrap root HTTP handler in debug.WithPprof(), rearrange router.Start() to support this
* remove unused code
* set debug buildtag in build script when $DEBUG set
* update go-debug version with fixed handler
* use clone of router.srv for LE cert manager, reset server timeouts in debug
* add kim's other libraries to README
The `database/sql` package in the Go stdlib has two methods that can run
queries against the database driver: `Query` and `Exec`. When the query
returns rows, such as "SELECT", the package expects the use of `Query`,
and returns `*sql.Rows`. When the query does not return rows, the
package expects the use of `Exec`, which returns `sql.Result`.
This changeset corrects the "media_attachments" migration to using
`ExecContext`, as "ALTER TABLE" does not return rows.
Signed-off-by: Terin Stock <terinjokes@gmail.com>
* start implementing trimming hook
* add test with very long test
* test syslog w/ unix socket + long (trimmed) msg
* trim long entries with trimhook
* trim to 1700 chars instead
