aboutsummaryrefslogtreecommitdiffhomepage
path: root/media
AgeCommit message (Collapse)Author
2024-06-04Add a HTTP cache for remote resources.Bjørn Erik Pedersen
Fixes #12502 Closes #11891
2024-05-14Create pages from _content.gotmplBjørn Erik Pedersen
Closes #12427 Closes #12485 Closes #6310 Closes #5074
2024-01-28all: Run gofumpt -l -w .Bjørn Erik Pedersen
2024-01-27all: Rework page store, add a dynacache, improve partial rebuilds, and some ↵develop2024Bjørn Erik Pedersen
general spring cleaning There are some breaking changes in this commit, see #11455. Closes #11455 Closes #11549 This fixes a set of bugs (see issue list) and it is also paying some technical debt accumulated over the years. We now build with Staticcheck enabled in the CI build. The performance should be about the same as before for regular sized Hugo sites, but it should perform and scale much better to larger data sets, as objects that uses lots of memory (e.g. rendered Markdown, big JSON files read into maps with transform.Unmarshal etc.) will now get automatically garbage collected if needed. Performance on partial rebuilds when running the server in fast render mode should be the same, but the change detection should be much more accurate. A list of the notable new features: * A new dependency tracker that covers (almost) all of Hugo's API and is used to do fine grained partial rebuilds when running the server. * A new and simpler tree document store which allows fast lookups and prefix-walking in all dimensions (e.g. language) concurrently. * You can now configure an upper memory limit allowing for much larger data sets and/or running on lower specced PCs. We have lifted the "no resources in sub folders" restriction for branch bundles (e.g. sections). Memory Limit * Hugos will, by default, set aside a quarter of the total system memory, but you can set this via the OS environment variable HUGO_MEMORYLIMIT (in gigabytes). This is backed by a partitioned LRU cache used throughout Hugo. A cache that gets dynamically resized in low memory situations, allowing Go's Garbage Collector to free the memory. New Dependency Tracker: Hugo has had a rule based coarse grained approach to server rebuilds that has worked mostly pretty well, but there have been some surprises (e.g. stale content). This is now revamped with a new dependency tracker that can quickly calculate the delta given a changed resource (e.g. a content file, template, JS file etc.). This handles transitive relations, e.g. $page -> js.Build -> JS import, or $page1.Content -> render hook -> site.GetPage -> $page2.Title, or $page1.Content -> shortcode -> partial -> site.RegularPages -> $page2.Content -> shortcode ..., and should also handle changes to aggregated values (e.g. site.Lastmod) effectively. This covers all of Hugo's API with 2 known exceptions (a list that may not be fully exhaustive): Changes to files loaded with template func os.ReadFile may not be handled correctly. We recommend loading resources with resources.Get Changes to Hugo objects (e.g. Page) passed in the template context to lang.Translate may not be detected correctly. We recommend having simple i18n templates without too much data context passed in other than simple types such as strings and numbers. Note that the cachebuster configuration (when A changes then rebuild B) works well with the above, but we recommend that you revise that configuration, as it in most situations should not be needed. One example where it is still needed is with TailwindCSS and using changes to hugo_stats.json to trigger new CSS rebuilds. Document Store: Previously, a little simplified, we split the document store (where we store pages and resources) in a tree per language. This worked pretty well, but the structure made some operations harder than they needed to be. We have now restructured it into one Radix tree for all languages. Internally the language is considered to be a dimension of that tree, and the tree can be viewed in all dimensions concurrently. This makes some operations re. language simpler (e.g. finding translations is just a slice range), but the idea is that it should also be relatively inexpensive to add more dimensions if needed (e.g. role). Fixes #10169 Fixes #10364 Fixes #10482 Fixes #10630 Fixes #10656 Fixes #10694 Fixes #10918 Fixes #11262 Fixes #11439 Fixes #11453 Fixes #11457 Fixes #11466 Fixes #11540 Fixes #11551 Fixes #11556 Fixes #11654 Fixes #11661 Fixes #11663 Fixes #11664 Fixes #11669 Fixes #11671 Fixes #11807 Fixes #11808 Fixes #11809 Fixes #11815 Fixes #11840 Fixes #11853 Fixes #11860 Fixes #11883 Fixes #11904 Fixes #7388 Fixes #7425 Fixes #7436 Fixes #7544 Fixes #7882 Fixes #7960 Fixes #8255 Fixes #8307 Fixes #8863 Fixes #8927 Fixes #9192 Fixes #9324
2023-12-18all: Remove unused codeBjørn Erik Pedersen
Using x/tools/cmd/deadcode
2023-08-07Add all config to docshelper.jsonBjørn Erik Pedersen
Also consolidate so the mediaTypes and outputFormats are listed once only. Fixes #11328
2023-05-16Create a struct with all of Hugo's config optionsBjørn Erik Pedersen
Primary motivation is documentation, but it will also hopefully simplify the code. Also, * Lower case the default output format names; this is in line with the custom ones (map keys) and how it's treated all the places. This avoids doing `stringds.EqualFold` everywhere. Closes #10896 Closes #10620
2023-03-02Correct typos in Go commentsOleksandr Redko
2023-03-01Replace deprecated ioutil with io and osOleksandr Redko
https://pkg.go.dev/io/ioutil is deprecated since Go 1.16.
2023-01-16Fix HEAD method in resources.GetRemoteBjørn Erik Pedersen
Fixes #10604
2023-01-04Misc doc, code refactoring to improve documentationBjørn Erik Pedersen
2022-10-26media: Rename application/javascript, application/typescript to ↵Bjørn Erik Pedersen
text/javascript etc. The former is deprecated. See https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/MIME_types/Common_types See https://discourse.gohugo.io/t/mime-type-for-javascript-is-now-text-javascript/40813
2022-05-30Add Markdown as an output formatBjørn Erik Pedersen
The motivation behind this is not to make it easier to publish Markdown files, as that sounds unusual. This is mainly meant for shortcodes that produces Markdown to be inlined. You would do this by creating shortcodes with `*.md` suffix (e.g. `layouts/shortcodes/myshortcode.md`). This output format is defined as plain text, and will use Go's much more lenient text template parser. Updates #9821
2022-04-27Some godoc adjustments and image struct renamesBjørn Erik Pedersen
2022-03-17all: gofmt -w -r 'interface{} -> any' .Bjørn Erik Pedersen
Updates #9687
2022-02-10Update the application/javascript media typeJoe Mooring
Add jsm and mjs extensions. Closes #9483
2021-12-22media: Also consider extension in FromContentBjørn Erik Pedersen
As used in `resources.GetRemote`. This will now reject image files with text and text files with images.
2021-12-22media: Add missing BMP and GIF to the default MediaTypes listBjørn Erik Pedersen
2021-12-22media: Add PDF MIME typeBjørn Erik Pedersen
2021-12-17Always use content to resolve content type in resources.GetRemoteBjørn Erik Pedersen
This is a security hardening measure; don't trust the URL extension or any `Content-Type`/`Content-Disposition` header on its own, always look at the file content using Go's `http.DetectContentType`. This commit also adds ttf and otf media type definitions to Hugo. Fixes #9302 Fixes #9301
2021-12-13Improve handling of remote image/jpeg resources (#9278)Joe Mooring
Add jpe, jif, and jfif to image/jpeg extensions. For remote image/jpeg without extension, always use jpg extension. Closes #9275
2021-12-03media: Add rss suffix for application/rss+xmlPaul van Brouwershaven
Closes #9233
2021-06-18media: Adjust test assertionBjørn Erik Pedersen
Closes #8625
2021-06-18media: support application/manifest+jsonRohan Kumar
The standard file extension for Web App Manifest files is ".webmanifest". This commit allows Hugo to recognize .webmanifest files as "application/manifest+json" files and to minify them using its JSON minifier. The .webmanifest file extension is recommended in the w3c spec to simplify media type registration: https://www.w3.org/TR/appmanifest/#media-type-registration Webhint docs are also relevant: https://webhint.io/docs/user-guide/hints/hint-manifest-file-extension/ Closes #8624
2021-06-14Misc config loading fixesBjørn Erik Pedersen
The main motivation behind this is simplicity and correctnes, but the new small config library is also faster: ``` BenchmarkDefaultConfigProvider/Viper-16 252418 4546 ns/op 2720 B/op 30 allocs/op BenchmarkDefaultConfigProvider/Custom-16 450756 2651 ns/op 1008 B/op 6 allocs/op ``` Fixes #8633 Fixes #8618 Fixes #8630 Updates #8591 Closes #6680 Closes #5192
2021-04-20Regression in media type suffix lookupBjørn Erik Pedersen
Introduced in Hugo 0.82.0. Fixes #8406
2021-04-15Add webp image encoding supportBjørn Erik Pedersen
Fixes #5924
2021-03-14media: Make Type comparableBjørn Erik Pedersen
So we can use it and output.Format as map key etc. This commit also fixes the media.Type implementation so it does not need to mutate itself to handle different suffixes for the same MIME type, e.g. jpg vs. jpeg. This means that there are no Suffix or FullSuffix on media.Type anymore. Fixes #8317 Fixes #8324
2021-03-11media: Add a basic benchmarkBjørn Erik Pedersen
2020-12-16all: Fix minor typosPhil Davis
2020-12-03all: Format code with gofumptBjørn Erik Pedersen
See https://github.com/mvdan/gofumpt
2020-07-22resources/js: Add option for setting bundle formatBjørn Erik Pedersen
Fixes #7503
2020-07-13Add proper Media Type handling in js.BuildBjørn Erik Pedersen
See #732
2020-03-20Some minify configuration adjustmentsBjørn Erik Pedersen
2019-12-19Fix incorrect MIME type from image/jpg to image/jpegJohn Zaitseff
According to multiple sources, both official ([IANA] and [RFC2046]) and otherwise (eg, [Mozilla] and [Wikipedia]), the official MIME type for JPEG images is `image/jpeg`, not `image/jpg`. Change Hugo to match. [IANA]: https://www.iana.org/assignments/media-types/media-types.xhtml#image [RFC2046]: https://tools.ietf.org/html/rfc2046 [Mozilla]: https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/MIME_types [Wikipedia]: https://en.wikipedia.org/wiki/JPEG
2019-10-16media: Add common video media typesNicolas Martignoni
- See https://discourse.gohugo.io/t/resources-bytype-for-video-files/21285
2019-10-06media: Use + to create the Type stringBjørn Erik Pedersen
2019-09-21resources: Support output image format in image operationsJ. Ansorg
The image format is defined as the image extension of the known formats, excluding the dot. All of 'img.Resize "600x jpeg"', 'img.Resize "600x jpg"', and 'img.Resize "600x png"' are valid format definitions. If the target format is defined in the operation definition string, then the converted image will be stored in this format. Permalinks and media type are updated correspondingly. Unknown image extensions in the operation definition have not effect. See #6298
2019-08-12tests: Convert from testify to quicktestBjørn Erik Pedersen
2019-03-23Make Page an interfaceBjørn Erik Pedersen
The main motivation of this commit is to add a `page.Page` interface to replace the very file-oriented `hugolib.Page` struct. This is all a preparation step for issue #5074, "pages from other data sources". But this also fixes a set of annoying limitations, especially related to custom output formats, and shortcodes. Most notable changes: * The inner content of shortcodes using the `{{%` as the outer-most delimiter will now be sent to the content renderer, e.g. Blackfriday. This means that any markdown will partake in the global ToC and footnote context etc. * The Custom Output formats are now "fully virtualized". This removes many of the current limitations. * The taxonomy list type now has a reference to the `Page` object. This improves the taxonomy template `.Title` situation and make common template constructs much simpler. See #5074 Fixes #5763 Fixes #5758 Fixes #5090 Fixes #5204 Fixes #4695 Fixes #5607 Fixes #5707 Fixes #5719 Fixes #3113 Fixes #5706 Fixes #5767 Fixes #5723 Fixes #5769 Fixes #5770 Fixes #5771 Fixes #5759 Fixes #5776 Fixes #5777 Fixes #5778
2018-12-23tpl/transform: Add transform.Unmarshal funcBjørn Erik Pedersen
Fixes #5428
2018-09-07media: Fix golint issuesCameron Moore
Fix godoc issues and the following: media/mediaType.go:228:1: receiver name t should be consistent with previous receiver name m for Type
2018-08-28Improve minifier MIME type resolutionBjørn Erik Pedersen
This commit also removes the deprecated `Suffix` from MediaType. Now use `Suffixes` and put the MIME type suffix in the type, e.g. `application/svg+xml`. Fixes #5093
2018-07-10media: Only show deprecation warning when neededBjørn Erik Pedersen
Fixes #4927
2018-07-10media: Allow multiple file suffixes per media typeBjørn Erik Pedersen
Before this commit, `Suffix` on `MediaType` was used both to set a custom file suffix and as a way to augment the mediatype definition (what you see after the "+", e.g. "image/svg+xml"). This had its limitations. For one, it was only possible with one file extension per MIME type. Now you can specify multiple file suffixes using "suffixes", but you need to specify the full MIME type identifier: [mediaTypes] [mediaTypes."image/svg+xml"] suffixes = ["svg", "abc ] In most cases, it will be enough to just change: [mediaTypes] [mediaTypes."my/custom-mediatype"] suffix = "txt" To: [mediaTypes] [mediaTypes."my/custom-mediatype"] suffixes = ["txt"] Hugo will still respect values set in "suffix" if no value for "suffixes" is provided, but this will be removed in a future release. Note that you can still get the Media Type's suffix from a template: {{ $mediaType.Suffix }}. But this will now map to the MIME type filename. Fixes #4920
2018-07-06Add Hugo Piper with SCSS support and much moreBjørn Erik Pedersen
Before this commit, you would have to use page bundles to do image processing etc. in Hugo. This commit adds * A new `/assets` top-level project or theme dir (configurable via `assetDir`) * A new template func, `resources.Get` which can be used to "get a resource" that can be further processed. This means that you can now do this in your templates (or shortcodes): ```bash {{ $sunset := (resources.Get "images/sunset.jpg").Fill "300x200" }} ``` This also adds a new `extended` build tag that enables powerful SCSS/SASS support with source maps. To compile this from source, you will also need a C compiler installed: ``` HUGO_BUILD_TAGS=extended mage install ``` Note that you can use output of the SCSS processing later in a non-SCSSS-enabled Hugo. The `SCSS` processor is a _Resource transformation step_ and it can be chained with the many others in a pipeline: ```bash {{ $css := resources.Get "styles.scss" | resources.ToCSS | resources.PostCSS | resources.Minify | resources.Fingerprint }} <link rel="stylesheet" href="{{ $styles.RelPermalink }}" integrity="{{ $styles.Data.Digest }}" media="screen"> ``` The transformation funcs above have aliases, so it can be shortened to: ```bash {{ $css := resources.Get "styles.scss" | toCSS | postCSS | minify | fingerprint }} <link rel="stylesheet" href="{{ $styles.RelPermalink }}" integrity="{{ $styles.Data.Digest }}" media="screen"> ``` A quick tip would be to avoid the fingerprinting part, and possibly also the not-superfast `postCSS` when you're doing development, as it allows Hugo to be smarter about the rebuilding. Documentation will follow, but have a look at the demo repo in https://github.com/bep/hugo-sass-test New functions to create `Resource` objects: * `resources.Get` (see above) * `resources.FromString`: Create a Resource from a string. New `Resource` transformation funcs: * `resources.ToCSS`: Compile `SCSS` or `SASS` into `CSS`. * `resources.PostCSS`: Process your CSS with PostCSS. Config file support (project or theme or passed as an option). * `resources.Minify`: Currently supports `css`, `js`, `json`, `html`, `svg`, `xml`. * `resources.Fingerprint`: Creates a fingerprinted version of the given Resource with Subresource Integrity.. * `resources.Concat`: Concatenates a list of Resource objects. Think of this as a poor man's bundler. * `resources.ExecuteAsTemplate`: Parses and executes the given Resource and data context (e.g. .Site) as a Go template. Fixes #4381 Fixes #4903 Fixes #4858
2017-09-29Clean up lint in various packagesCameron Moore
Changes fall into one of the following: - gofmt -s - receiver name is inconsistent - omit unused 2nd value from range - godoc comment formed incorrectly - err assigned and not used - if block ends with a return statement followed by else
2017-08-07all: Fix spellingBjørn Erik Pedersen
And some other minor issues.
2017-08-07all: gofmt -sBjørn Erik Pedersen
2017-08-03Add some missing doc commentsJorin Vogel
As pointed out by the linter, some exported functions and types are missing doc comments. The linter warnings have been reduced from 194 to 116. Not all missing comments have been added in this commit though.
generated by cgit v1.2.3 (git 2.39.1) at 2024-12-26 14:41:47 +0000