Commit graph

6 commits

Author SHA1 Message Date
kim cde2fb6244
[feature] support processing of (many) more media types (#3090)
* initial work replacing our media decoding / encoding pipeline with ffprobe + ffmpeg

* specify the video codec to use when generating static image from emoji

* update go-storage library (fixes incompatibility after updating go-iotools)

* maintain image aspect ratio when generating a thumbnail for it

* update readme to show go-ffmpreg

* fix a bunch of media tests, move filesize checking to callers of media manager for more flexibility

* remove extra debug from error message

* fix up incorrect function signatures

* update PutFile to just use regular file copy, as changes are file is on separate partition

* fix remaining tests, remove some unneeded tests now we're working with ffmpeg/ffprobe

* update more tests, add more code comments

* add utilities to generate processed emoji / media outputs

* fix remaining tests

* add test for opus media file, add license header to utility cmds

* limit the number of concurrently available ffmpeg / ffprobe instances

* reduce number of instances

* further reduce number of instances

* fix envparsing test with configuration variables

* update docs and configuration with new media-{local,remote}-max-size variables
2024-07-12 09:39:47 +00:00
kim 5318054808
[performance] media processing improvements (#1288)
* media processor consolidation and reformatting, reduce amount of required syscalls

Signed-off-by: kim <grufwub@gmail.com>

* update go-store library, stream jpeg/png encoding + use buffer pools, improved media processing AlreadyExists error handling

Signed-off-by: kim <grufwub@gmail.com>

* fix duration not being set, fix mp4 test expecting error

Signed-off-by: kim <grufwub@gmail.com>

* fix test expecting media files with different extension

Signed-off-by: kim <grufwub@gmail.com>

* remove unused code

Signed-off-by: kim <grufwub@gmail.com>

* fix expected storage paths in tests, update expected test thumbnails

Signed-off-by: kim <grufwub@gmail.com>

* remove dead code

Signed-off-by: kim <grufwub@gmail.com>

* fix cached presigned s3 url fetching

Signed-off-by: kim <grufwub@gmail.com>

* fix tests

Signed-off-by: kim <grufwub@gmail.com>

* fix test models

Signed-off-by: kim <grufwub@gmail.com>

* update media processing to use sync.Once{} for concurrency protection

Signed-off-by: kim <grufwub@gmail.com>

* shutup linter

Signed-off-by: kim <grufwub@gmail.com>

* fix passing in KVStore GetStream() as stream to PutStream()

Signed-off-by: kim <grufwub@gmail.com>

* fix unlocks of storage keys

Signed-off-by: kim <grufwub@gmail.com>

* whoops, return the error...

Signed-off-by: kim <grufwub@gmail.com>

* pour one out for tobi's code <3

Signed-off-by: kim <grufwub@gmail.com>

* add back the byte slurping code

Signed-off-by: kim <grufwub@gmail.com>

* check for both ErrUnexpectedEOF and EOF

Signed-off-by: kim <grufwub@gmail.com>

* add back links to file format header information

Signed-off-by: kim <grufwub@gmail.com>

Signed-off-by: kim <grufwub@gmail.com>
2023-01-11 12:13:13 +01:00
tobi 91c8d5d20d
[bugfix] Fix thumbnails not taking exif rotation into account (#746)
* use disintegration/imaging instead of nfnt/resize

* update tests

* use disintegration lib for thumbing (if necessary)
2022-08-10 14:05:14 +02:00
tobi ddfd83d0fb
update media processing (#244) 2021-09-23 11:13:11 +02:00
Tobi Smethurst 6cd033449f
Refine statuses (#26)
Remote media is now dereferenced and attached properly to incoming federated statuses.
    Mentions are now dereferenced and attached properly to incoming federated statuses.
    Small fixes to status visibility.
    Allow URL params for filtering statuses:

	// ExcludeRepliesKey is for specifying whether to exclude replies in a list of returned statuses by an account.
      	// PinnedKey is for specifying whether to include pinned statuses in a list of returned statuses by an account.
      	// MaxIDKey is for specifying the maximum ID of the status to retrieve.
      	// MediaOnlyKey is for specifying that only statuses with media should be returned in a list of returned statuses by an account.

    Add endpoint for fetching an account's statuses.
2021-05-17 19:06:58 +02:00
Tobi Smethurst 71a49e2b43
Api/v1/accounts (#8)
* start work on accounts module

* plodding away on the accounts endpoint

* groundwork for other account routes

* add password validator

* validation utils

* require account approval flags

* comments

* comments

* go fmt

* comments

* add distributor stub

* rename api to federator

* tidy a bit

* validate new account requests

* rename r router

* comments

* add domain blocks

* add some more shortcuts

* add some more shortcuts

* check email + username availability

* email block checking for signups

* chunking away at it

* tick off a few more things

* some fiddling with tests

* add mock package

* relocate repo

* move mocks around

* set app id on new signups

* initialize oauth server properly

* rename oauth server

* proper mocking tests

* go fmt ./...

* add required fields

* change name of func

* move validation to account.go

* more tests!

* add some file utility tools

* add mediaconfig

* new shortcut

* add some more fields

* add followrequest model

* add notify

* update mastotypes

* mock out storage interface

* start building media interface

* start on update credentials

* mess about with media a bit more

* test image manipulation

* media more or less working

* account update nearly working

* rearranging my package ;) ;) ;)

* phew big stuff!!!!

* fix type checking

* *fiddles*

* Add CreateTables func

* account registration flow working

* tidy

* script to step through auth flow

* add a lil helper for generating user uris

* fiddling with federation a bit

* update progress

* Tidying and linting
2021-04-01 20:46:45 +02:00