aboutsummaryrefslogtreecommitdiffhomepage
path: root/deps
AgeCommit message (Collapse)Author
9 daysjs: Fix js.Batch for multihost setupsBjørn Erik Pedersen
Note that this is an unreleased feature. Fixes #13151
12 daysWrite all logging (INFO, WARN, ERROR) to stderrBjørn Erik Pedersen
The old setup tried to log >= warning to stderr, the rest to stdout. However, that logic was flawed, so warnings ended up in stdout, which makes `hugo list all` etc. hard to reason about from scripts. This commit fixes this by making all logging (info, warn, error) log to stderr and let stdout be reserved for program output. Fixes #13074
13 daysAdd js.BatchBjørn Erik Pedersen
Fixes #12626 Closes #7499 Closes #9978 Closes #12879 Closes #13113 Fixes #13116
2024-10-20Make sure that HugoSites is always closed when doneBjørn Erik Pedersen
Including all the integration tests.
2024-08-09Add build time math renderingBjørn Erik Pedersen
While very useful on its own (and combined with the passthrough render hooks), this also serves as a proof of concept of using WASI (WebAssembly System Interface) modules in Hugo. This will be marked _experimental_ in the documentation. Not because it will be removed or changed in a dramatic way, but we need to think a little more how to best set up/configure similar services, define where these WASM files gets stored, maybe we can allow user provided WASM files plugins via Hugo Modules mounts etc. See these issues for more context: * https://github.com/gohugoio/hugo/issues/12736 * https://github.com/gohugoio/hugo/issues/12737 See #11927
2024-06-25Add css.TailwindCSSBjørn Erik Pedersen
Closes #12618 Closes #12620
2024-06-23Implement deferBjørn Erik Pedersen
Closes #8086 Closes #12589
2024-06-04Add a HTTP cache for remote resources.Bjørn Erik Pedersen
Fixes #12502 Closes #11891
2024-04-25Fix rebuilds when running hugo -wBjørn Erik Pedersen
This was partly broken in Hugo 0.123.0. We have two internal config options that gets set from the CLI: * Running; a web server is running * Watching; either set via `hugo -w` or `hugo server --watch=false` Part of the change detection code wrongly used the `Running` as a flag when `Watching` would be the correct. Fixes #12296
2024-03-15all: Typo fixesChristian Oliff
2024-01-27all: Rework page store, add a dynacache, improve partial rebuilds, and some ↵develop2024Bjørn Erik Pedersen
general spring cleaning There are some breaking changes in this commit, see #11455. Closes #11455 Closes #11549 This fixes a set of bugs (see issue list) and it is also paying some technical debt accumulated over the years. We now build with Staticcheck enabled in the CI build. The performance should be about the same as before for regular sized Hugo sites, but it should perform and scale much better to larger data sets, as objects that uses lots of memory (e.g. rendered Markdown, big JSON files read into maps with transform.Unmarshal etc.) will now get automatically garbage collected if needed. Performance on partial rebuilds when running the server in fast render mode should be the same, but the change detection should be much more accurate. A list of the notable new features: * A new dependency tracker that covers (almost) all of Hugo's API and is used to do fine grained partial rebuilds when running the server. * A new and simpler tree document store which allows fast lookups and prefix-walking in all dimensions (e.g. language) concurrently. * You can now configure an upper memory limit allowing for much larger data sets and/or running on lower specced PCs. We have lifted the "no resources in sub folders" restriction for branch bundles (e.g. sections). Memory Limit * Hugos will, by default, set aside a quarter of the total system memory, but you can set this via the OS environment variable HUGO_MEMORYLIMIT (in gigabytes). This is backed by a partitioned LRU cache used throughout Hugo. A cache that gets dynamically resized in low memory situations, allowing Go's Garbage Collector to free the memory. New Dependency Tracker: Hugo has had a rule based coarse grained approach to server rebuilds that has worked mostly pretty well, but there have been some surprises (e.g. stale content). This is now revamped with a new dependency tracker that can quickly calculate the delta given a changed resource (e.g. a content file, template, JS file etc.). This handles transitive relations, e.g. $page -> js.Build -> JS import, or $page1.Content -> render hook -> site.GetPage -> $page2.Title, or $page1.Content -> shortcode -> partial -> site.RegularPages -> $page2.Content -> shortcode ..., and should also handle changes to aggregated values (e.g. site.Lastmod) effectively. This covers all of Hugo's API with 2 known exceptions (a list that may not be fully exhaustive): Changes to files loaded with template func os.ReadFile may not be handled correctly. We recommend loading resources with resources.Get Changes to Hugo objects (e.g. Page) passed in the template context to lang.Translate may not be detected correctly. We recommend having simple i18n templates without too much data context passed in other than simple types such as strings and numbers. Note that the cachebuster configuration (when A changes then rebuild B) works well with the above, but we recommend that you revise that configuration, as it in most situations should not be needed. One example where it is still needed is with TailwindCSS and using changes to hugo_stats.json to trigger new CSS rebuilds. Document Store: Previously, a little simplified, we split the document store (where we store pages and resources) in a tree per language. This worked pretty well, but the structure made some operations harder than they needed to be. We have now restructured it into one Radix tree for all languages. Internally the language is considered to be a dimension of that tree, and the tree can be viewed in all dimensions concurrently. This makes some operations re. language simpler (e.g. finding translations is just a slice range), but the idea is that it should also be relatively inexpensive to add more dimensions if needed (e.g. role). Fixes #10169 Fixes #10364 Fixes #10482 Fixes #10630 Fixes #10656 Fixes #10694 Fixes #10918 Fixes #11262 Fixes #11439 Fixes #11453 Fixes #11457 Fixes #11466 Fixes #11540 Fixes #11551 Fixes #11556 Fixes #11654 Fixes #11661 Fixes #11663 Fixes #11664 Fixes #11669 Fixes #11671 Fixes #11807 Fixes #11808 Fixes #11809 Fixes #11815 Fixes #11840 Fixes #11853 Fixes #11860 Fixes #11883 Fixes #11904 Fixes #7388 Fixes #7425 Fixes #7436 Fixes #7544 Fixes #7882 Fixes #7960 Fixes #8255 Fixes #8307 Fixes #8863 Fixes #8927 Fixes #9192 Fixes #9324
2023-10-20tpl/debug: Add debug.TimerBjørn Erik Pedersen
Closes #11580
2023-06-18Replace the old log setup, with structured logging etc.Bjørn Erik Pedersen
Fixes #11124
2023-05-28Fix Processed images count regression for multiple languagesBjørn Erik Pedersen
Fixes #11002
2023-05-18all: Fix comments for exported functions and packagesOleksandr Redko
2023-05-16Create a struct with all of Hugo's config optionsBjørn Erik Pedersen
Primary motivation is documentation, but it will also hopefully simplify the code. Also, * Lower case the default output format names; this is in line with the custom ones (map keys) and how it's treated all the places. This avoids doing `stringds.EqualFold` everywhere. Closes #10896 Closes #10620
2023-04-13Prevent the global error collector to panic when sending on closed channelBjørn Erik Pedersen
2023-03-04Page context handling in i18nBjørn Erik Pedersen
This is a workaround. We need to improve on this, but not today. Fixes #10782
2023-01-25Only invoke a given cached partial onceBjørn Erik Pedersen
Note that this is backed by a LRU cache (which we soon shall see more usage of), so if you're a heavy user of cached partials it may be evicted and refreshed if needed. But in most cases every partial is only invoked once. This commit also adds a timeout (the global `timeout` config option) to make infinite recursion in partials easier to reason about. ``` name old time/op new time/op delta IncludeCached-10 8.92ms ± 0% 8.48ms ± 1% -4.87% (p=0.016 n=4+5) name old alloc/op new alloc/op delta IncludeCached-10 6.65MB ± 0% 5.17MB ± 0% -22.32% (p=0.002 n=6+6) name old allocs/op new allocs/op delta IncludeCached-10 117k ± 0% 71k ± 0% -39.44% (p=0.002 n=6+6) ``` Closes #4086 Updates #9588
2022-09-26Fix /static performance regression from Hugo 0.103.0Bjørn Erik Pedersen
In `v0.103.0` we added support for `resources.PostProcess` for all file types, not just HTML. We had benchmarks that said we were fine in that department, but those did not consider the static file syncing. This fixes that by: * Making sure that the /static syncer always gets its own file system without any checks for the post process token. * For dynamic files (e.g. rendered HTML files) we add an additional check to make sure that we skip binary files (e.g. images) Fixes #10328
2022-09-14Support PostProcess for all file typesBjørn Erik Pedersen
Not just HTML. Fixes #10269
2022-05-06Improve error messages, esp. when the server is runningBjørn Erik Pedersen
* Add file context to minifier errors when publishing * Misc fixes (see issues) * Allow custom server error template in layouts/server/error.html To get to this, this commit also cleans up and simplifies the code surrounding errors and files. This also removes the usage of `github.com/pkg/errors`, mostly because of https://github.com/pkg/errors/issues/223 -- but also because most of this is now built-in to Go. Fixes #9852 Fixes #9857 Fixes #9863
2022-03-17all: gofmt -w -r 'interface{} -> any' .Bjørn Erik Pedersen
Updates #9687
2021-12-16Add some basic security policies with sensible defaultsBjørn Erik Pedersen
This ommmit contains some security hardening measures for the Hugo build runtime. There are some rarely used features in Hugo that would be good to have disabled by default. One example would be the "external helpers". For `asciidoctor` and some others we use Go's `os/exec` package to start a new process. These are a predefined set of binary names, all loaded from `PATH` and with a predefined set of arguments. Still, if you don't use `asciidoctor` in your project, you might as well have it turned off. You can configure your own in the new `security` configuration section, but the defaults are configured to create a minimal amount of site breakage. And if that do happen, you will get clear instructions in the loa about what to do. The default configuration is listed below. Note that almost all of these options are regular expression _whitelists_ (a string or a slice); the value `none` will block all. ```toml [security] enableInlineShortcodes = false [security.exec] allow = ['^dart-sass-embedded$', '^go$', '^npx$', '^postcss$'] osEnv = ['(?i)^(PATH|PATHEXT|APPDATA|TMP|TEMP|TERM)$'] [security.funcs] getenv = ['^HUGO_'] [security.http] methods = ['(?i)GET|POST'] urls = ['.*'] ```
2021-10-16Reimplement archetypesBjørn Erik Pedersen
The old implementation had some issues, mostly related to the context (e.g. name, file paths) passed to the template. This new implementation is using the exact same code path for evaluating the pages as in a regular build. This also makes it more robust and easier to reason about in a multilingual setup. Now, if you are explicit about the target path, Hugo will now always pick the correct mount and language: ```bash hugo new content/en/posts/my-first-post.md ``` Fixes #9032 Fixes #7589 Fixes #9043 Fixes #9046 Fixes #9047
2021-06-07tpl/fmt: Add erroridf template funcBjørn Erik Pedersen
Fixes #8613
2020-12-30Add Dart Sass supportBjørn Erik Pedersen
But note that the Dart Sass Embedded Protocol is still in beta (beta 5), a main release scheduled for Q1 2021. Fixes #7380 Fixes #8102
2020-12-03all: Format code with gofumptBjørn Erik Pedersen
See https://github.com/mvdan/gofumpt
2020-11-03Make js.Build fully support modulesBjørn Erik Pedersen
Fixes #7816 Fixes #7777 Fixes #7916
2020-10-22Allow getJSON errors to be ignoredBjørn Erik Pedersen
This change is mostly motivated to get a more stable CI build (we're building the Hugo site there, with Instagram and Twitter shortcodes sometimes failing). Fixes #7866
2020-09-29langs/i18n: Upgrade to go-i18n v2Bjørn Erik Pedersen
Fixes #5242
2020-09-07Fix some change detection issues on server reloadsBjørn Erik Pedersen
* Fix change detection when .GetPage/site.GetPage is used from shortcode * Fix stale content for GetPage results with short name lookups on server reloads Fixes #7623 Fixes #7624 Fixes #7625
2020-04-07Add basic "post resource publish support"Bjørn Erik Pedersen
Fixes #7146
2020-03-11Improve Tailwind/PostCSS error messagesBjørn Erik Pedersen
Fixes #7041 Fixes #7042
2020-03-09Fix handling of HTML files without front matterBjørn Erik Pedersen
This means that any HTML file inside /content will be treated as a regular file. If you want it processes with shortcodes and a layout, add front matter. The defintion of an HTML file here is: * File with extension .htm or .html * With first non-whitespace character "<" that isn't a HTML comment. This is in line with the documentation. Fixes #7030 Fixes #7028 See #6789
2020-01-23deps: Make the build flags shared between sitesBjørn Erik Pedersen
Updates #6789
2020-01-22tpl/tplimpl: Rework template management to get rid of concurrency issuesBjørn Erik Pedersen
This more or less completes the simplification of the template handling code in Hugo started in v0.62. The main motivation was to fix a long lasting issue about a crash in HTML content files without front matter. But this commit also comes with a big functional improvement. As we now have moved the base template evaluation to the build stage we now use the same lookup rules for `baseof` as for `list` etc. type of templates. This means that in this simple example you can have a `baseof` template for the `blog` section without having to duplicate the others: ``` layouts ├── _default │   ├── baseof.html │   ├── list.html │   └── single.html └── blog └── baseof.html ``` Also, when simplifying code, you often get rid of some double work, as shown in the "site building" benchmarks below. These benchmarks looks suspiciously good, but I have repeated the below with ca. the same result. Compared to master: ``` name old time/op new time/op delta SiteNew/Bundle_with_image-16 13.1ms ± 1% 10.5ms ± 1% -19.34% (p=0.029 n=4+4) SiteNew/Bundle_with_JSON_file-16 13.0ms ± 0% 10.7ms ± 1% -18.05% (p=0.029 n=4+4) SiteNew/Tags_and_categories-16 46.4ms ± 2% 43.1ms ± 1% -7.15% (p=0.029 n=4+4) SiteNew/Canonify_URLs-16 52.2ms ± 2% 47.8ms ± 1% -8.30% (p=0.029 n=4+4) SiteNew/Deep_content_tree-16 77.9ms ± 1% 70.9ms ± 1% -9.01% (p=0.029 n=4+4) SiteNew/Many_HTML_templates-16 43.0ms ± 0% 37.2ms ± 1% -13.54% (p=0.029 n=4+4) SiteNew/Page_collections-16 58.2ms ± 1% 52.4ms ± 1% -9.95% (p=0.029 n=4+4) name old alloc/op new alloc/op delta SiteNew/Bundle_with_image-16 3.81MB ± 0% 2.22MB ± 0% -41.70% (p=0.029 n=4+4) SiteNew/Bundle_with_JSON_file-16 3.60MB ± 0% 2.01MB ± 0% -44.20% (p=0.029 n=4+4) SiteNew/Tags_and_categories-16 19.3MB ± 1% 14.1MB ± 0% -26.91% (p=0.029 n=4+4) SiteNew/Canonify_URLs-16 70.7MB ± 0% 69.0MB ± 0% -2.40% (p=0.029 n=4+4) SiteNew/Deep_content_tree-16 37.1MB ± 0% 31.2MB ± 0% -15.94% (p=0.029 n=4+4) SiteNew/Many_HTML_templates-16 17.6MB ± 0% 10.6MB ± 0% -39.92% (p=0.029 n=4+4) SiteNew/Page_collections-16 25.9MB ± 0% 21.2MB ± 0% -17.99% (p=0.029 n=4+4) name old allocs/op new allocs/op delta SiteNew/Bundle_with_image-16 52.3k ± 0% 26.1k ± 0% -50.18% (p=0.029 n=4+4) SiteNew/Bundle_with_JSON_file-16 52.3k ± 0% 26.1k ± 0% -50.16% (p=0.029 n=4+4) SiteNew/Tags_and_categories-16 336k ± 1% 269k ± 0% -19.90% (p=0.029 n=4+4) SiteNew/Canonify_URLs-16 422k ± 0% 395k ± 0% -6.43% (p=0.029 n=4+4) SiteNew/Deep_content_tree-16 401k ± 0% 313k ± 0% -21.79% (p=0.029 n=4+4) SiteNew/Many_HTML_templates-16 247k ± 0% 143k ± 0% -42.17% (p=0.029 n=4+4) SiteNew/Page_collections-16 282k ± 0% 207k ± 0% -26.55% (p=0.029 n=4+4) ``` Fixes #6716 Fixes #6760 Fixes #6768 Fixes #6778
2019-12-12Rework template handling for function and map lookupsBjørn Erik Pedersen
This is a big commit, but it deletes lots of code and simplifies a lot. * Resolving the template funcs at execution time means we don't have to create template clones per site * Having a custom map resolver means that we can remove the AST lower case transformation for the special lower case Params map Not only is the above easier to reason about, it's also faster, especially if you have more than one language, as in the benchmark below: ``` name old time/op new time/op delta SiteNew/Deep_content_tree-16 53.7ms ± 0% 48.1ms ± 2% -10.38% (p=0.029 n=4+4) name old alloc/op new alloc/op delta SiteNew/Deep_content_tree-16 41.0MB ± 0% 36.8MB ± 0% -10.26% (p=0.029 n=4+4) name old allocs/op new allocs/op delta SiteNew/Deep_content_tree-16 481k ± 0% 410k ± 0% -14.66% (p=0.029 n=4+4) ``` This should be even better if you also have lots of templates. Closes #6594
2019-11-06Prepare for GoldmarkBjørn Erik Pedersen
This commmit prepares for the addition of Goldmark as the new Markdown renderer in Hugo. This introduces a new `markup` package with some common interfaces and each implementation in its own package. See #5963
2019-07-25Block symlink dir traversal for /staticBjørn Erik Pedersen
This is in line with how it behaved before, but it was lifted a little for the project mount for Hugo Modules, but that could create hard-to-detect loops.
2019-07-24Add Hugo ModulesBjørn Erik Pedersen
This commit implements Hugo Modules. This is a broad subject, but some keywords include: * A new `module` configuration section where you can import almost anything. You can configure both your own file mounts nd the file mounts of the modules you import. This is the new recommended way of configuring what you earlier put in `configDir`, `staticDir` etc. And it also allows you to mount folders in non-Hugo-projects, e.g. the `SCSS` folder in the Bootstrap GitHub project. * A module consists of a set of mounts to the standard 7 component types in Hugo: `static`, `content`, `layouts`, `data`, `assets`, `i18n`, and `archetypes`. Yes, Theme Components can now include content, which should be very useful, especially in bigger multilingual projects. * Modules not in your local file cache will be downloaded automatically and even "hot replaced" while the server is running. * Hugo Modules supports and encourages semver versioned modules, and uses the minimal version selection algorithm to resolve versions. * A new set of CLI commands are provided to manage all of this: `hugo mod init`, `hugo mod get`, `hugo mod graph`, `hugo mod tidy`, and `hugo mod vendor`. All of the above is backed by Go Modules. Fixes #5973 Fixes #5996 Fixes #6010 Fixes #5911 Fixes #5940 Fixes #6074 Fixes #6082 Fixes #6092
2019-03-26Avoid nilpointer on no File on PageBjørn Erik Pedersen
Fixes #5781
2019-03-23Make Page an interfaceBjørn Erik Pedersen
The main motivation of this commit is to add a `page.Page` interface to replace the very file-oriented `hugolib.Page` struct. This is all a preparation step for issue #5074, "pages from other data sources". But this also fixes a set of annoying limitations, especially related to custom output formats, and shortcodes. Most notable changes: * The inner content of shortcodes using the `{{%` as the outer-most delimiter will now be sent to the content renderer, e.g. Blackfriday. This means that any markdown will partake in the global ToC and footnote context etc. * The Custom Output formats are now "fully virtualized". This removes many of the current limitations. * The taxonomy list type now has a reference to the `Page` object. This improves the taxonomy template `.Title` situation and make common template constructs much simpler. See #5074 Fixes #5763 Fixes #5758 Fixes #5090 Fixes #5204 Fixes #4695 Fixes #5607 Fixes #5707 Fixes #5719 Fixes #3113 Fixes #5706 Fixes #5767 Fixes #5723 Fixes #5769 Fixes #5770 Fixes #5771 Fixes #5759 Fixes #5776 Fixes #5777 Fixes #5778
2019-01-02resources: Move resource interfaces into its own packageBjørn Erik Pedersen
2018-12-23tpl/transform: Add transform.Unmarshal funcBjørn Erik Pedersen
Fixes #5428
2018-12-06Add tpl/site and tpl/hugoBjørn Erik Pedersen
This means that the current `.Site` and ´.Hugo` is available as a globals, so you can do `site.IsServer`, `hugo.Version` etc. Fixes #5470 Fixes #5467 Fixes #5503
2018-11-23Handle themes in the new file cache (for images, assets)Bjørn Erik Pedersen
In the newly consolidated file cache implementation, we forgot that we also look in the theme(s) for assets (SCSS transformations etc.), which is not good for Netlify and the demo sites. Fixes #5460
2018-11-13Add a consolidated file cacheBjørn Erik Pedersen
This commits reworks how file caching is performed in Hugo. Now there is only one way, and it can be configured. This is the default configuration: ```toml [caches] [caches.getjson] dir = ":cacheDir" maxAge = -1 [caches.getcsv] dir = ":cacheDir" maxAge = -1 [caches.images] dir = ":resourceDir/_gen" maxAge = -1 [caches.assets] dir = ":resourceDir/_gen" maxAge = -1 ``` You can override any of these cache setting in your own `config.toml`. The placeholders explained: `:cacheDir`: This is the value of the `cacheDir` config option if set (can also be set via OS env variable `HUGO_CACHEDIR`). It will fall back to `/opt/build/cache/hugo_cache/` on Netlify, or a `hugo_cache` directory below the OS temp dir for the others. `:resourceDir`: This is the value of the `resourceDir` config option. `maxAge` is the time in seconds before a cache entry will be evicted, -1 means forever and 0 effectively turns that particular cache off. This means that if you run your builds on Netlify, all caches configured with `:cacheDir` will be saved and restored on the next build. For other CI vendors, please read their documentation. For an CircleCI example, see https://github.com/bep/hugo-sass-test/blob/6c3960a8f4b90e8938228688bc49bdcdd6b2d99e/.circleci/config.yml Fixes #5404
2018-10-22hugolib: Continue the file context/line number errors workBjørn Erik Pedersen
See #5324
2018-10-16commands: Show server error info in browserBjørn Erik Pedersen
The main item in this commit is showing of errors with a file context when running `hugo server`. This can be turned off: `hugo server --disableBrowserError` (can also be set in `config.toml`). But to get there, the error handling in Hugo needed a revision. There are some items left TODO for commits soon to follow, most notable errors in content and config files. Fixes #5284 Fixes #5290 See #5325 See #5324