Commit graph

24 commits

Author SHA1 Message Date
kim 223025fc27
[security] transport.Controller{} and transport.Transport{} security and performance improvements ()
* cache transports in controller by privkey-generated pubkey, add retry logic to transport requests

Signed-off-by: kim <grufwub@gmail.com>

* update code comments, defer mutex unlocks

Signed-off-by: kim <grufwub@gmail.com>

* add count to 'performing request' log message

Signed-off-by: kim <grufwub@gmail.com>

* reduce repeated conversions of same url.URL object

Signed-off-by: kim <grufwub@gmail.com>

* move worker.Worker to concurrency subpackage, add WorkQueue type, limit transport http client use by WorkQueue

Signed-off-by: kim <grufwub@gmail.com>

* fix security advisories regarding max outgoing conns, max rsp body size

- implemented by a new httpclient.Client{} that wraps an underlying
  client with a queue to limit connections, and limit reader wrapping
  a response body with a configured maximum size
- update pub.HttpClient args passed around to be this new httpclient.Client{}

Signed-off-by: kim <grufwub@gmail.com>

* add httpclient tests, move ip validation to separate package + change mechanism

Signed-off-by: kim <grufwub@gmail.com>

* fix merge conflicts

Signed-off-by: kim <grufwub@gmail.com>

* use singular mutex in transport rather than separate signer mus

Signed-off-by: kim <grufwub@gmail.com>

* improved useragent string

Signed-off-by: kim <grufwub@gmail.com>

* add note regarding missing test

Signed-off-by: kim <grufwub@gmail.com>

* remove useragent field from transport (instead store in controller)

Signed-off-by: kim <grufwub@gmail.com>

* shutup linter

Signed-off-by: kim <grufwub@gmail.com>

* reset other signing headers on each loop iteration

Signed-off-by: kim <grufwub@gmail.com>

* respect request ctx during retry-backoff sleep period

Signed-off-by: kim <grufwub@gmail.com>

* use external pkg with docs explaining performance "hack"

Signed-off-by: kim <grufwub@gmail.com>

* use http package constants instead of string method literals

Signed-off-by: kim <grufwub@gmail.com>

* add license file headers

Signed-off-by: kim <grufwub@gmail.com>

* update code comment to match new func names

Signed-off-by: kim <grufwub@gmail.com>

* updates to user-agent string

Signed-off-by: kim <grufwub@gmail.com>

* update signed testrig models to fit with new transport logic (instead uses separate signer now)

Signed-off-by: kim <grufwub@gmail.com>

* fuck you linter

Signed-off-by: kim <grufwub@gmail.com>
2022-05-15 11:16:43 +02:00
kim 420e2fb22b
replace async client API / federator msg processing with worker pools ()
* replace async client API / federator msg processing with worker pools
* appease our lord-and-saviour, the linter
2022-04-28 13:23:11 +01:00
tsmethurst 3301148bb7 merge fixup 2022-02-08 12:17:53 +01:00
tsmethurst 4c294a596a Merge branch 'main' into media_refactor 2022-02-08 12:17:18 +01:00
tobi 5c9d20cea3
don't bother negotiating Accept for webfinger () 2022-02-06 15:00:57 +01:00
tobi 1b36e85840
[feature] Rework timeline code to make it useful for more than just statuses ()
* add preparable and timelineable interfaces

* initialize timeline manager within the processor

* generic renaming

* move status-specific timeline logic into the processor

* refactor timeline to make it useful for more than statuses
2022-02-05 12:47:38 +01:00
tsmethurst e0f9323b9a test the media manager a bit, add shutdown logic 2022-01-10 18:36:09 +01:00
tsmethurst c4d63d125b more refactoring, media handler => manager 2021-12-28 16:36:00 +01:00
tobi ef5a9256a8
Extend license notices to 2022 () 2021-12-20 18:42:19 +01:00
tobi cb8688f429
Remove unnecessary storage config variables ()
* rewire config to not use extraneous serve vars

* rename 'file' to 'local' for consistency

* use Type and Size again
2021-12-20 15:19:53 +01:00
tobi e2daf0f012
Add Accept header negotiation to relevant API endpoints ()
* start centralizing negotiation logic for API

* swagger document nodeinfo endpoint

* go fmt

* document negotiate function

* use content negotiation

* tidy up negotiation logic

* negotiate content throughout client api

* swagger

* remove attachment on Content

* add accept header to test requests
2021-12-11 17:50:00 +01:00
tobi 0884f89431
Implement Cobra CLI tooling, Viper config tooling ()
* start pulling out + replacing urfave and config

* replace many many instances of config

* move more stuff => viper

* properly remove urfave

* move some flags to root command

* add testrig commands to root

* alias config file keys

* start adding cli parsing tests

* reorder viper init

* remove config path alias

* fmt

* change config file keys to non-nested

* we're more or less in business now

* tidy up the common func

* go fmt

* get tests passing again

* add note about the cliparsing tests

* reorganize

* update docs with changes

* structure cmd dir better

* rename + move some files around

* fix dangling comma
2021-12-07 13:31:39 +01:00
tobi ce22e03f9d
Require confirmed email when checking oauth token ()
* move token checker to security package

* update tests with new security package

* add oauth token checking to security package

* check if user email confirmed when parsing token
2021-11-27 14:53:34 +01:00
tobi 829a934d23
update dependencies () 2021-11-13 12:29:08 +01:00
tobi 2aaec82732
smtp + email confirmation ()
* add smtp configuration

* add email confirm + reset templates

* add email sender to testrig

* flesh out the email sender interface

* go fmt

* golint

* update from field with more clarity

* tidy up the email formatting

* fix tests

* add email sender to processor

* tidy client api processing a bit

* further tidying in fromClientAPI

* pin new account to user

* send msg to processor on new account creation

* generate confirm email uri

* remove emailer from account processor again

* add processCreateAccountFromClientAPI

* move emailer accountprocessor => userprocessor

* add email sender to user processor

* SendConfirmEmail function

* add noop email sender

* use noop email sender in tests

* only assemble message if callback is not nil

* use noop email sender if no smtp host is defined

* minify email html before sending

* fix wrong email address

* email confirm test

* fmt

* serve web hndler

* add email confirm handler

* init test log properly on testrig

* log emails that *would* have been sent

* go fmt ./...

* unexport confirm email handler

* updatedAt

* test confirm email function

* don't allow tokens older than 7 days

* change error message a bit

* add basic smtp docs

* add a few more snippets

* typo

* add email sender to outbox tests

* don't use dutch wikipedia link

* don't minify email html
2021-10-31 15:46:23 +01:00
R. Aidan Campbell 083099a957
reference global logrus ()
* reference logrus' global logger instead of passing and storing a logger reference everywhere

* always directly use global logrus logger instead of referencing an instance

* test suites should also directly use the global logrus logger

* rename gin logging function to clarify that it's middleware

* correct comments which erroneously referenced removed logger parameter

* setting log level for tests now uses logrus' exported type instead of the string value, to guarantee error isn't possible
2021-10-11 14:37:33 +02:00
tobi b315eee43b
Update webfingering a little, add tests ()
* Update webfingering a little, add tests

* fix broken tests oops
2021-09-20 16:46:45 +02:00
Tobi Smethurst 113186ce4e
lil webfingy fix ()
* lil webfingy fix

* return requested content type from fede requests
2021-07-21 17:46:59 +02:00
Tobi Smethurst b1a4f38e38
allow different host + accountDomain ()
* allow different host + accountDomain

* use accountDomain in tags
2021-07-19 18:42:08 +02:00
Tobi Smethurst d389e7b150
Domain block ()
* start work on admin domain blocking

* move stuff around + further work on domain blocks

* move + restructure processor

* prep work for deleting account

* tidy

* go fmt

* formatting

* domain blocking more work

* check domain blocks way earlier on

* progress on delete account

* delete more stuff when an account is gone

* and more...

* domain blocky block block

* get individual domain block, delete a block
2021-07-05 13:23:03 +02:00
Tobi Smethurst b4288f3c47
Timeline manager ()
* start messing about with timeline manager

* i have no idea what i'm doing

* i continue to not know what i'm doing

* it's coming along

* bit more progress

* update timeline with new posts as they come in

* lint and fmt

* Select accounts where empty string

* restructure a bunch, get unfaves working

* moving stuff around

* federate status deletes properly

* mention regex better but not 100% there

* fix regex

* some more hacking away at the timeline code phew

* fix up some little things

* i can't even

* more timeline stuff

* move to ulid

* fiddley

* some lil fixes for kibou compatibility

* timelines working pretty alright!

* tidy + lint
2021-06-13 18:42:28 +02:00
Tobi Smethurst 3d77f81c7f
Move a lot of stuff + tidy stuff ()
Lots of renaming and moving stuff, some bug fixes, more lenient parsing of notifications and home timeline.
2021-05-30 13:12:00 +02:00
Tobi Smethurst cc48294c31
Inbox post ()
Inbox POST from federated servers now working for statuses and follow requests.
    Follow request client API added.
    Start work on federating outgoing messages.
    Other fixes and changes/tidying up.
2021-05-15 11:58:11 +02:00
Tobi Smethurst dc338dc881
Webfinger + Small fixes () 2021-05-09 20:34:27 +02:00