Commit graph

121 commits

Author SHA1 Message Date
kim b8879ac68a
[dependencies] update go-store, go-mutexes ()
* update go-store, go-mutexes

Signed-off-by: kim <grufwub@gmail.com>

* update vendored code

Signed-off-by: kim <grufwub@gmail.com>
2022-03-08 12:56:53 +01:00
tobi 07727753b9
[feature] Clean up/uncache remote media ()
* Add whereNotEmptyAndNotNull

* Add GetRemoteOlderThanDays

* Add GetRemoteOlderThanDays

* Add PruneRemote to Manager interface

* Start implementing PruneRemote

* add new attachment + status to tests

* fix up and test GetRemoteOlderThan

* fix bad import

* PruneRemote: return number pruned

* add Cached column to mediaattachment

* update + test pruneRemote

* update mediaTest

* use Cached column

* upstep bun to latest version

* embed structs in mediaAttachment

* migrate mediaAttachment to new format

* don't default cached to true

* select only remote media

* update db dependencies

* step bun back to last working version

* update pruneRemote to use Cached field

* fix storage path of test attachments

* add recache logic to manager

* fix trimmed aspect ratio

* test prune and recache

* return errwithcode

* tidy up different paths for emoji vs attachment

* fix incorrect thumbnail type being stored

* expose TransportController to media processor

* implement tee-ing recached content

* add thoughts of dog to test fedi attachments

* test get remote files

* add comment on PruneRemote

* add postData cleanup to recache

* test thumbnail fetching

* add incredible diagram

* go mod tidy

* buffer pipes for recache streaming

* test for client stops reading after 1kb

* add media-remote-cache-days to config

* add cron package

* wrap logrus so it's available to cron

* start and stop cron jobs gracefully
2022-03-07 11:08:26 +01:00
tobi a089a98ea9
[feature] Gin enable gzip encoding ()
* add gin gzip dependency

* add gzip middleware to router

* go mod tidy
2022-02-19 12:12:41 +01:00
tsmethurst a766b9265b Go mod tidy 2022-02-08 13:39:47 +01:00
tsmethurst 4e74c84148 update go-store to latest 2022-01-29 12:15:51 +01:00
tsmethurst 9aa364f1eb go mod tidy 2022-01-25 14:33:23 +01:00
tsmethurst f28cf793ee upgrade go-store 2022-01-24 17:35:13 +01:00
tsmethurst 7d024ce74d use exif-terminator 2022-01-23 14:41:31 +01:00
tsmethurst 6f5ccf4355 update dependencies 2022-01-16 18:52:30 +01:00
tsmethurst e08c0e55ee add gruf worker pool 2022-01-03 17:37:09 +01:00
kim 635ad2a42f
Update codeberg.org/gruf libraries and fix go-store issue ()
* update codeberg.org/gruf/ libraries

Signed-off-by: kim <grufwub@gmail.com>

* another update

Signed-off-by: kim <grufwub@gmail.com>
2021-12-20 10:35:32 +01:00
tobi c111b239f7
Add optional syslog logrus hook ()
* add optional syslog logrus hook

* document syslog
2021-12-12 18:00:20 +01:00
tobi 67ac8db190
Upstep Go dependencies ()
* Upstep Go dependencies

* tiny linter fix

* Tidy
2021-12-12 15:47:51 +01:00
tobi 0884f89431
Implement Cobra CLI tooling, Viper config tooling ()
* start pulling out + replacing urfave and config

* replace many many instances of config

* move more stuff => viper

* properly remove urfave

* move some flags to root command

* add testrig commands to root

* alias config file keys

* start adding cli parsing tests

* reorder viper init

* remove config path alias

* fmt

* change config file keys to non-nested

* we're more or less in business now

* tidy up the common func

* go fmt

* get tests passing again

* add note about the cliparsing tests

* reorganize

* update docs with changes

* structure cmd dir better

* rename + move some files around

* fix dangling comma
2021-12-07 13:31:39 +01:00
tobi 182b4eea73
Update dependencies () 2021-11-27 15:26:58 +01:00
tobi 09ef9e639e
move to ssb gofed fork () 2021-11-13 17:29:43 +01:00
tobi 829a934d23
update dependencies () 2021-11-13 12:29:08 +01:00
tobi 8b7c3507fe
upstep bun to v1.0.14 () 2021-10-24 13:14:37 +02:00
tobi 9a53b1a8d1
upstep bun to v1.0.9 () 2021-09-29 15:09:45 +02:00
tobi c7cfbe2702
Unblock fix ()
* start tests for inbox posts

* go mod tidy

* rename transferContext

* test block/unblock

* improve logging

* improve logging

* fix comment typo
2021-09-24 17:56:48 +02:00
tobi 142f37f1bd
upstep bun version () 2021-09-23 11:13:28 +02:00
kim 635281f133
update my personal library versions ()
Signed-off-by: kim (grufwub) <grufwub@gmail.com>
2021-09-13 10:33:01 +02:00
kim (grufwub) e43a46e982 add git.iim.gay/grufwub/go-store for storage backend, replacing blob.Storage
Signed-off-by: kim (grufwub) <grufwub@gmail.com>
2021-09-11 20:12:47 +01:00
kim bac4ee9980
update bun library to latest commit ()
* update bun library to latest commit

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* update to latest bun release

Signed-off-by: kim (grufwub) <grufwub@gmail.com>
2021-09-11 13:53:44 +02:00
tobi 64bd689e55
Test both dbs ()
* move scripts, allow testing both dbs with one cmd

* tidy + vendor

* update test.sh to ignore cache

* put test commands directly in drone.yml

* change CONTRIBUTING slightly

* go ham on the readme
2021-09-10 18:13:24 +02:00
tobi f2e5bedea6
migrate go version to 1.17 ()
* migrate go version to 1.17

* update contributing
2021-09-10 14:42:14 +02:00
tobi 555ea8edfb
Import export ()
* start with export/import code

* messing about with decoding/encoding

* some more fiddling

* stuff is WORKING

* working pretty alright!

* go fmt

* fix up tests, add docs

* start backup/restore doc

* tweaks

* credits

* update advancedVisibility settings

* update bun library -> v1.0.4

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* update oauth library -> v4.3.1-SSB

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* handle oauth token scope, fix user.SigninCount + token.UserID

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* update oauth library --> v4.3.2-SSB

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* update sqlite library -> v1.13.0

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* review changes

* start with export/import code

* messing about with decoding/encoding

* some more fiddling

* stuff is WORKING

* working pretty alright!

* go fmt

* fix up tests, add docs

* start backup/restore doc

* tweaks

* credits

* update advancedVisibility settings

* review changes

Co-authored-by: kim (grufwub) <grufwub@gmail.com>
Co-authored-by: kim <89579420+NyaaaWhatsUpDoc@users.noreply.github.com>
2021-09-09 16:15:25 +02:00
kim a027da0ac9
Merge pull request from NyaaaWhatsUpDoc/update/sqlite-library
Update/sqlite library
2021-09-08 21:26:32 +01:00
kim (grufwub) 6a2d0d9508 Merge remote-tracking branch 'upstream/main' into update/sqlite-library
Signed-off-by: kim (grufwub) <grufwub@gmail.com>
2021-09-08 21:13:54 +01:00
kim (grufwub) 71a4f8667c update sqlite library -> v1.13.0
Signed-off-by: kim (grufwub) <grufwub@gmail.com>
2021-09-08 21:12:23 +01:00
kim (grufwub) 151069cff4 Merge remote-tracking branch 'upstream/main' into update/oauth-library
Signed-off-by: kim (grufwub) <grufwub@gmail.com>
2021-09-08 20:55:13 +01:00
kim (grufwub) 0ea131fd7b update oauth library --> v4.3.2-SSB
Signed-off-by: kim (grufwub) <grufwub@gmail.com>
2021-09-08 20:51:42 +01:00
kim (grufwub) fce3ba6382 update oauth library -> v4.3.1-SSB
Signed-off-by: kim (grufwub) <grufwub@gmail.com>
2021-09-08 20:20:06 +01:00
kim (grufwub) bdcc090851 update bun library -> v1.0.4
Signed-off-by: kim (grufwub) <grufwub@gmail.com>
2021-09-08 20:05:26 +01:00
kim ed46224573
Add SQLite support, fix un-thread-safe DB caches, small performance f… ()
* Add SQLite support, fix un-thread-safe DB caches, small performance fixes

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* add SQLite licenses to README

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* appease the linter, and fix my dumbass-ery

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* make requested changes

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* add back comment

Signed-off-by: kim (grufwub) <grufwub@gmail.com>
2021-08-29 16:41:41 +02:00
tobi 2dc9fc1626
Pg to bun ()
* start moving to bun

* changing more stuff

* more

* and yet more

* tests passing

* seems stable now

* more big changes

* small fix

* little fixes
2021-08-25 15:34:33 +02:00
tobi 071eca20ce
Manually approves followers ()
* update go-fed

* update go-fed

* manuallyapprovesfollowers

* serialize manuallyApprovesFollowers
2021-08-23 12:46:05 +02:00
tobi 4920229a3b
Database updates ()
* start moving some database stuff around

* continue moving db stuff around

* more fiddling

* more updates

* and some more

* and yet more

* i broke SOMETHING but what, it's a mystery

* tidy up

* vendor ttlcache

* use ttlcache

* fix up some tests

* rename some stuff

* little reminder

* some more updates
2021-08-20 12:26:56 +02:00
Tobi Smethurst ce190d867c
Text/status parsing fixes ()
* aaaaaa

* vendor minify

* update + test markdown parsing
2021-08-16 19:17:56 +02:00
Tobi Smethurst ea8ad8b346
Link parsing ()
* add link parsing + formatting functionality

* refinement + docs

* add missing test

* credit url library
2021-07-28 11:42:26 +02:00
Tobi Smethurst ad0e26dc04
Markdown Statuses ()
* parse markdown statuses if desired

* add some preliminary docs for writing posts
2021-07-26 20:25:54 +02:00
Tobi Smethurst e2757ae676
add trusted proxy for parsing client IPs () 2021-07-26 16:15:36 +02:00
Tobi Smethurst 05e9af089c
Oidc ()
* add oidc config

* inching forward with oidc idp

* lil webfingy fix

* bit more progress

* further oidc

* oidc now working

* document dex config

* replace broken images

* add additional credits

* tiny doc update

* update

* add oidc config

* inching forward with oidc idp

* bit more progress

* further oidc

* oidc now working

* document dex config

* replace broken images

* add additional credits

* tiny doc update

* update

* document

* docs + comments
2021-07-23 10:36:28 +02:00
Tobi Smethurst 677490bc4e
Db tls ()
* go mod tidy

* complete example config

* add tls support for db connection

* add certpool to tlsConfig

* add some lil docker scripts
2021-07-19 18:03:07 +02:00
Tobi Smethurst bdba3ff9a9
sanitize html for statuses + instance ()
* sanitize html for statuses + instance

* sanitization
2021-07-13 16:03:51 +02:00
f0x52 f9bc305aca
new styling for frontpage, update login and authorize templates ()
* new styling for frontpage, update login and authorize templates

* run go fmt

* add AssetBaseDir to command flag parsing

* untested: move landing page to it's own router

* go fmt, fix typo

* fix package, adapt to proper Route structure
2021-06-21 19:46:10 +02:00
Tobi Smethurst b4288f3c47
Timeline manager ()
* start messing about with timeline manager

* i have no idea what i'm doing

* i continue to not know what i'm doing

* it's coming along

* bit more progress

* update timeline with new posts as they come in

* lint and fmt

* Select accounts where empty string

* restructure a bunch, get unfaves working

* moving stuff around

* federate status deletes properly

* mention regex better but not 100% there

* fix regex

* some more hacking away at the timeline code phew

* fix up some little things

* i can't even

* more timeline stuff

* move to ulid

* fiddley

* some lil fixes for kibou compatibility

* timelines working pretty alright!

* tidy + lint
2021-06-13 18:42:28 +02:00
Tobi Smethurst 3d77f81c7f
Move a lot of stuff + tidy stuff ()
Lots of renaming and moving stuff, some bug fixes, more lenient parsing of notifications and home timeline.
2021-05-30 13:12:00 +02:00
Tobi Smethurst cc48294c31
Inbox post ()
Inbox POST from federated servers now working for statuses and follow requests.
    Follow request client API added.
    Start work on federating outgoing messages.
    Other fixes and changes/tidying up.
2021-05-15 11:58:11 +02:00
Tobi Smethurst 71a49e2b43
Api/v1/accounts ()
* start work on accounts module

* plodding away on the accounts endpoint

* groundwork for other account routes

* add password validator

* validation utils

* require account approval flags

* comments

* comments

* go fmt

* comments

* add distributor stub

* rename api to federator

* tidy a bit

* validate new account requests

* rename r router

* comments

* add domain blocks

* add some more shortcuts

* add some more shortcuts

* check email + username availability

* email block checking for signups

* chunking away at it

* tick off a few more things

* some fiddling with tests

* add mock package

* relocate repo

* move mocks around

* set app id on new signups

* initialize oauth server properly

* rename oauth server

* proper mocking tests

* go fmt ./...

* add required fields

* change name of func

* move validation to account.go

* more tests!

* add some file utility tools

* add mediaconfig

* new shortcut

* add some more fields

* add followrequest model

* add notify

* update mastotypes

* mock out storage interface

* start building media interface

* start on update credentials

* mess about with media a bit more

* test image manipulation

* media more or less working

* account update nearly working

* rearranging my package ;) ;) ;)

* phew big stuff!!!!

* fix type checking

* *fiddles*

* Add CreateTables func

* account registration flow working

* tidy

* script to step through auth flow

* add a lil helper for generating user uris

* fiddling with federation a bit

* update progress

* Tidying and linting
2021-04-01 20:46:45 +02:00
Tobi Smethurst aa9ce272dc
Oauth/token ()
* add host and protocol options

* some fiddling

* tidying up and comments

* tick off /oauth/token

* tidying a bit

* tidying

* go mod tidy

* allow attaching middleware to server

* add middleware

* more user friendly

* add comments

* comments

* store account + app

* tidying

* lots of restructuring

* lint + tidy
2021-03-22 22:26:54 +01:00
tsmethurst 3388c870ae Go mod tidy 2021-03-20 19:08:25 +01:00
tsmethurst 1b11884121 auth flow working for code 2021-03-18 23:27:43 +01:00
tsmethurst 6eab00e05e getting there....... 2021-03-17 16:01:31 +01:00
tsmethurst 9d5fb0785f fiddling 2021-03-17 13:14:52 +01:00
tsmethurst eb2ff2ab23 Some more messing around with oauth2 2021-03-17 11:33:06 +01:00
tsmethurst 4e281f31b0 whole buncha stuff 2021-03-15 23:05:24 +01:00
tsmethurst b48072fef6 go mod tidy 2021-03-15 19:00:14 +01:00
tsmethurst cca676dcb3 tests 2021-03-15 18:59:38 +01:00
tsmethurst 1eecc2688c bit of experimenting and tidying 2021-03-15 16:15:14 +01:00
tsmethurst 772f6e5949 start working on oauth2 implementation 2021-03-14 17:56:16 +01:00
tsmethurst c558681f02 start adding mastodon api types 2021-03-11 14:30:14 +01:00
tsmethurst 59963090cb create db schemas for accounts 2021-03-05 18:31:12 +01:00
tsmethurst 18d0685ef1 further fun 2021-03-03 21:15:20 +01:00
tsmethurst 54c4b8de20 further yak shaving 2021-03-03 18:12:02 +01:00
tsmethurst b8e0f33c35 start the service using a cli 2021-03-03 11:28:28 +01:00
tsmethurst be1b631681 More messing around 2021-03-02 22:52:31 +01:00
tsmethurst d8e8d44d47 start implementing db interface 2021-03-02 18:26:30 +01:00
tsmethurst c879e1b0b3 Messing around a bit 2021-03-02 12:46:48 +01:00
tsmethurst e68b72ddf9 add docs and a bunch of other stuff 2021-02-28 15:17:18 +01:00
tsmethurst fbf52fe84b messing around a bit 2021-02-27 22:57:50 +01:00