Code Monkey home page Code Monkey logo

Comments (16)

vgwidt avatar vgwidt commented on August 11, 2024

If using pre-compiled binaries don't forget to install wget:

RUN apt-get update && apt-get install -y wget && rm -rf /var/lib/apt/lists/*

from sumi.

vgwidt avatar vgwidt commented on August 11, 2024

Threw together a sample that doesn't cache cargo dependencies yet.

FROM rust:latest as base
RUN apt-get update && apt-get install -y wget && rm -rf /var/lib/apt/lists/*
RUN wget -qO- https://github.com/thedodd/trunk/releases/download/v0.16.0/trunk-x86_64-unknown-linux-gnu.tar.gz | tar -xzf- && chmod +x trunk && cp trunk /usr/local/cargo/bin/
RUN rustup target add wasm32-unknown-unknown
RUN wget -qO- https://github.com/rustwasm/wasm-bindgen/releases/download/0.2.84/wasm-bindgen-0.2.84-aarch64-unknown-linux-gnu.tar.gz | tar -xzf- && cp ./wasm-bindgen-0.2.84-aarch64-unknown-linux-gnu/wasm-bindgen /usr/local/cargo/bin/ && chmod +x /usr/local/cargo/bin/wasm-bindgen

FROM base as builder
WORKDIR /usr/src/sumi
COPY . .
RUN trunk build -d dist ./frontend/index.html --release
RUN cargo build --manifest-path ./backend/Cargo.toml --release

FROM rust:latest
WORKDIR /usr/src/sumi
RUN apt-get install -y libpq-dev
RUN cargo install diesel_cli --no-default-features --features postgres
COPY --from=builder /usr/src/sumi/target/release/sumi-backend ./sumi-backend

I'm not sure it is actually worth avoiding using cargo to install trunk and wasm-bindgen as that bit should be cached.

from sumi.

vgwidt avatar vgwidt commented on August 11, 2024

Sample with cargo-chef for caching project (so far only backend) dependencies:

FROM rust:latest as base
RUN cargo install cargo-chef
RUN cargo install trunk
RUN rustup target add wasm32-unknown-unknown

FROM base as planner
WORKDIR /usr/src/sumi
COPY . .
RUN cargo chef prepare --recipe-path recipe.json

FROM base as cacher
WORKDIR /usr/src/sumi
COPY --from=planner /usr/src/sumi/recipe.json recipe.json
RUN cargo chef cook --release --recipe-path recipe.json

FROM base as builder
WORKDIR /usr/src/sumi
COPY . .
COPY --from=cacher /usr/src/sumi/target target
RUN trunk build -d dist ./frontend/index.html --release
RUN cargo build --manifest-path ./backend/Cargo.toml --release

FROM rust:latest
WORKDIR /usr/src/sumi
RUN apt-get install -y libpq-dev
RUN cargo install diesel_cli --no-default-features --features postgres
COPY ./backend/diesel.toml ./backend/diesel.toml
COPY ./certificates ./certificates
COPY ./dist ./dist
COPY ./backend/migrations ./backend/migrations
COPY --from=builder /usr/src/sumi/target/release/sumi-backend ./sumi

This builds trunk and wasm-bindgen as that stage will be cached anyways. Only the backend dependencies are cached; not sure how to handle frontend yet. Using the Rust image and building Diesel results in a 1.5GB image as there are no precompiled binaries readily available (diesel 2379).

from sumi.

vgwidt avatar vgwidt commented on August 11, 2024

COPY --from=builder /usr/src/sumi/target/release/sumi-backend ./sumi doesn't appear to be working.

from sumi.

vgwidt avatar vgwidt commented on August 11, 2024

COPY --from=builder /usr/src/sumi/target/release/sumi-backend ./sumi doesn't appear to be working.

Dist folder wasn't being copied from the builder, need to change COPY ./dist ./dist to COPY --from=builder /usr/src/sumi/dist ./dist.

Need to fix up .dockerignore as well, and can just copy . . on final stage. It'll copy unused source files but they use minimal space.

from sumi.

vgwidt avatar vgwidt commented on August 11, 2024

A related issue is that some .env variables are still required for building the frontend.

from sumi.

vgwidt avatar vgwidt commented on August 11, 2024

Merged 7ac1b16 which includes the new Dockerfile. It currently only caches the backend.

Environment variables are now written to the index.html file when the backend is run. I'll need to figure out a solution that allows de-coupling of the API and web server, though that is currently not possible at this point anyways.

from sumi.

vgwidt avatar vgwidt commented on August 11, 2024

The issue with caching the frontend has to do with Cargo workspace limitations.
We can tell the frontend to target wasm32-unknown-unknown by adding the following in .cargo/config.toml:

[build]
target = "wasm32-unknown-unknown"

However, when building from the root workspace folder, it does not look at this config file for members. Alternatively, Cargo nightly allows us to force the package target in the member Cargo.toml: rust-lang/cargo#9406. I tested it and it works great, but I don't want to depend on nightly.

It appears that the only other option is to move frontend out of the workspace (would need to do an exclude or eliminate workspace altogether).

from sumi.

vgwidt avatar vgwidt commented on August 11, 2024

It looks like I might be able to build frontend deps using config.toml while building in the frontend folder. Sample (not working) Dockerfile that might demonstrate how it could be done when building with Docker:

FROM rust:latest as base
RUN cargo install cargo-chef
RUN cargo install trunk
RUN rustup target add wasm32-unknown-unknown

FROM base as cacher
WORKDIR /usr/src/sumi
COPY ./Cargo.lock ./Cargo.lock
COPY ./frontend/Cargo.toml ./frontend/Cargo.toml
COPY ./backend/Cargo.toml ./backend/Cargo.toml
COPY ./frontend/.cargo ./frontend/.cargo
RUN mkdir ./backend/src && mkdir ./frontend/src && echo 'fn main() { println!("Dummy"); }' > ./backend/src/main.rs && echo 'fn main() { println!("Dummy"); } ' > ./frontend/src/lib.rs
RUN cargo build --release --manifest-path ./backend/Cargo.toml
WORKDIR /usr/src/sumi/frontend
RUN cargo build --release
WORKDIR /usr/src/sumi
RUN rm -rf ./backend/src && rm -rf ./frontend/src

FROM base as builder
WORKDIR /usr/src/sumi
COPY . .
COPY --from=cacher /usr/src/sumi/target target
RUN touch -a -m ./backend/src/main.rs && touch -a -m ./frontend/src/lib.rs
RUN trunk build -d dist ./frontend/index.html --release
RUN cargo build --manifest-path ./backend/Cargo.toml --release

FROM rust:latest
WORKDIR /usr/src/sumi
RUN apt-get install -y libpq-dev
RUN cargo install diesel_cli --no-default-features --features postgres
COPY . .
COPY --from=builder /usr/src/sumi/dist ./dist
COPY --from=builder /usr/src/sumi/target/release/sumi-backend ./sumi

from sumi.

vgwidt avatar vgwidt commented on August 11, 2024

Missed root Cargo.file in last comment. The following works and built in less than 2 minutes with everything cached:

FROM rust:latest as base
RUN cargo install trunk
RUN rustup target add wasm32-unknown-unknown

FROM base as cacher
WORKDIR /usr/src/sumi
COPY ./Cargo.lock ./Cargo.lock
COPY ./Cargo.toml ./Cargo.toml
COPY ./frontend/Cargo.toml ./frontend/Cargo.toml
COPY ./backend/Cargo.toml ./backend/Cargo.toml
COPY ./frontend/.cargo ./frontend/.cargo
RUN mkdir ./backend/src && mkdir ./frontend/src && echo 'fn main() { println!("Dummy"); }' > ./backend/src/main.rs && echo 'fn main() { println!("Dummy"); } ' > ./frontend/src/lib.rs
RUN cargo build --release --manifest-path ./backend/Cargo.toml
WORKDIR /usr/src/sumi/frontend
RUN cargo build --release
WORKDIR /usr/src/sumi
RUN rm -rf ./backend/src && rm -rf ./frontend/src

FROM base as builder
WORKDIR /usr/src/sumi
COPY --from=cacher /usr/src/sumi/target target
COPY . .
RUN touch -a -m ./backend/src/main.rs && touch -a -m ./frontend/src/lib.rs
RUN trunk build -d dist ./frontend/index.html --release
RUN cargo build --manifest-path ./backend/Cargo.toml --release

FROM rust:latest
WORKDIR /usr/src/sumi
RUN apt-get install -y libpq-dev
RUN cargo install diesel_cli --no-default-features --features postgres
COPY . .
COPY --from=builder /usr/src/sumi/dist ./dist
COPY --from=builder /usr/src/sumi/target/release/sumi-backend ./sumi

I attempted to get it working the same way with cargo-chef without success. I may try it again as it is a pretty clean solution.

It can be cleaned up a bit (i.e. remove RUN rm).

from sumi.

vgwidt avatar vgwidt commented on August 11, 2024

Slightly cleaned up version without touch, as I don't think we need it:

FROM rust:latest as base
RUN cargo install cargo-chef
RUN cargo install trunk
RUN rustup target add wasm32-unknown-unknown

FROM base as cacher
WORKDIR /usr/src/sumi
COPY ./Cargo.lock ./
COPY ./Cargo.toml ./
COPY ./frontend/Cargo.toml ./frontend/Cargo.toml
COPY ./backend/Cargo.toml ./backend/Cargo.toml
COPY ./frontend/.cargo ./frontend/.cargo
RUN mkdir ./backend/src && mkdir ./frontend/src && echo 'fn main() { println!("Dummy"); }' > ./backend/src/main.rs && echo 'fn main() { println!("Dummy"); } ' > ./frontend/src/lib.rs
RUN cargo build --release --manifest-path ./backend/Cargo.toml
WORKDIR /usr/src/sumi/frontend
RUN cargo build --release

FROM base as builder
WORKDIR /usr/src/sumi
COPY --from=cacher /usr/src/sumi/target target
COPY . .
RUN trunk build -d dist ./frontend/index.html --release
RUN cargo build --manifest-path ./backend/Cargo.toml --release

FROM rust:latest
WORKDIR /usr/src/sumi
RUN apt-get install -y libpq-dev
RUN cargo install diesel_cli --no-default-features --features postgres
COPY . .
COPY --from=builder /usr/src/sumi/dist ./dist
COPY --from=builder /usr/src/sumi/target/release/sumi-backend ./sumi

It would be nice to not have to rebuild the both if only one was changed. Is this because Docker ignores cache after change was detected at COPY . .?

from sumi.

vgwidt avatar vgwidt commented on August 11, 2024

Touch is needed otherwise it may use the builds made in the cacher stage. Removed cargo-chef.

I'm pretty satisfied with the current state, but I still would like to get a smaller final image. That would require building diesel_cli by ourselves.

FROM rust:latest as base
RUN cargo install trunk
RUN rustup target add wasm32-unknown-unknown

FROM base as cacher
WORKDIR /usr/src/sumi
COPY ./Cargo.lock ./
COPY ./Cargo.toml ./
COPY ./frontend/Cargo.toml ./frontend/Cargo.toml
COPY ./backend/Cargo.toml ./backend/Cargo.toml
COPY ./frontend/.cargo ./frontend/.cargo
RUN mkdir ./backend/src && mkdir ./frontend/src && echo 'fn main() { println!("Dummy"); }' > ./backend/src/main.rs && echo 'fn main() { println!("Dummy"); } ' > ./frontend/src/lib.rs
RUN cargo build --release --manifest-path ./backend/Cargo.toml
WORKDIR /usr/src/sumi/frontend
RUN cargo build --release

FROM base as builder
WORKDIR /usr/src/sumi
COPY . .
COPY --from=cacher /usr/src/sumi/target target
RUN touch -a -m ./backend/src/main.rs && touch -a -m ./frontend/src/lib.rs
RUN trunk build -d dist ./frontend/index.html --release
RUN cargo build --manifest-path ./backend/Cargo.toml --release

FROM rust:latest
WORKDIR /usr/src/sumi
RUN apt-get install -y libpq-dev
RUN cargo install diesel_cli --no-default-features --features postgres
COPY . .
COPY --from=builder /usr/src/sumi/dist ./dist
COPY --from=builder /usr/src/sumi/target/release/sumi-backend ./sumi

from sumi.

vgwidt avatar vgwidt commented on August 11, 2024

Addition of shared broke the Dockerbuild, fixed in 5d7778c

from sumi.

vgwidt avatar vgwidt commented on August 11, 2024

https://docs.rs/diesel_migrations/latest/diesel_migrations/macro.embed_migrations.html can be used in lieu of diesel_cli. It reads the migrations at compile time.

from sumi.

vgwidt avatar vgwidt commented on August 11, 2024

Dependency caching is not working very well. I find myself rebuilding them most of the time without any changes to Cargo.toml or Cargo.lock.

from sumi.

vgwidt avatar vgwidt commented on August 11, 2024

In case I want to continue to use diesel_cli but build in builder stage:

In builder:

RUN cargo install diesel_cli --no-default-features --features postgres

In final image:

RUN apt-get -y update
RUN apt-get install -y libpq-dev
COPY --from=builder /usr/local/cargo/bin/diesel /usr/bin/diesel

libssl.so.1.1 doesn't exist in the bookworm image, but it does in the Rust, so we need to either move it or build it. It might look like this:

RUN apt-get install -y wget
RUN wget https://www.openssl.org/source/openssl-1.1.1o.tar.gz && tar -zxvf openssl-1.1.1o.tar.gz
WORKDIR ./openssl-1.1.1o
RUN ./config && make && make test
RUN mv libcrypto.so.1.1 /usr/bin/ && mv libssl.so.1.1 /usr/bin/

from sumi.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.