[{"data":1,"prerenderedAt":491},["ShallowReactive",2],{"docs-\u002Fdocs\u002Fself-hosting":3,"docs-nav":486},{"id":4,"title":5,"body":6,"description":477,"extension":478,"meta":479,"navigation":480,"path":481,"section":482,"seo":483,"stem":484,"__hash__":485},"docs\u002Fdocs\u002F02.self-hosting.md","Self-hosting",{"type":7,"value":8,"toc":467},"minimark",[9,13,18,21,78,105,112,115,120,123,167,173,176,198,209,212,271,293,297,304,325,329,352,355,370,374,380,417,420,424,431,463],[10,11,12],"p",{},"dummie is self-hostable end to end. There are two things to stand up: the control\nplane, which is a container like any other, and at least one host that can boot\nmicroVMs, which needs hardware virtualisation.",[14,15,17],"h2",{"id":16},"the-control-plane","The control plane",[10,19,20],{},"The control plane ships as a single image. It needs PostgreSQL for its own\nrecords, ClickHouse for event history, and an S3-compatible store for user\nartifacts — all three are in the compose file in the repository root:",[22,23,28],"pre",{"className":24,"code":25,"language":26,"meta":27,"style":27},"language-sh shiki shiki-themes github-light github-dark","git clone https:\u002F\u002Fgithub.com\u002Fcodingcoffee\u002Fdummie\ncd dummie\ndocker compose -f docker-compose.prod.yml up -d\n","sh","",[29,30,31,47,57],"code",{"__ignoreMap":27},[32,33,36,40,44],"span",{"class":34,"line":35},"line",1,[32,37,39],{"class":38},"sScJk","git",[32,41,43],{"class":42},"sZZnC"," clone",[32,45,46],{"class":42}," https:\u002F\u002Fgithub.com\u002Fcodingcoffee\u002Fdummie\n",[32,48,50,54],{"class":34,"line":49},2,[32,51,53],{"class":52},"sj4cs","cd",[32,55,56],{"class":42}," dummie\n",[32,58,60,63,66,69,72,75],{"class":34,"line":59},3,[32,61,62],{"class":38},"docker",[32,64,65],{"class":42}," compose",[32,67,68],{"class":52}," -f",[32,70,71],{"class":42}," docker-compose.prod.yml",[32,73,74],{"class":42}," up",[32,76,77],{"class":52}," -d\n",[10,79,80,81,84,85,88,89,92,93,96,97,100,101,104],{},"The ",[29,82,83],{},"control"," service reads its configuration from ",[29,86,87],{},"control\u002F.env",", and the three\nbacking services read theirs from ",[29,90,91],{},".postgres.env",", ",[29,94,95],{},".clickhouse.env"," and\n",[29,98,99],{},".rustfs.env",". Create those before the first ",[29,102,103],{},"up","; the compose file does not\ncarry defaults, on purpose — there are no credentials committed to this\nrepository for you to forget to change.",[10,106,107,108,111],{},"The API and the console both come up on port ",[29,109,110],{},"1323",".",[10,113,114],{},"Migrations are embedded in the binary rather than applied from a directory, so\nthere is no separate migration step to run and no way for the schema to drift\nfrom the code that expects it.",[116,117,119],"h3",{"id":118},"clickhouse-users","ClickHouse users",[10,121,122],{},"Two different things talk to ClickHouse, over two different protocols, with two\ndifferent sets of credentials. Keeping them separate is deliberate.",[124,125,126,147],"ul",{},[127,128,129,132,133,136,137,139,140,143,144,111],"li",{},[130,131,17],"strong",{}," reads event history to answer the console's panels, and\napplies the ClickHouse migrations on startup. It uses ",[29,134,135],{},"CLICKHOUSE_URL"," from\n",[29,138,87],{}," — the native protocol, port ",[29,141,142],{},"9000"," — and because it owns the\nschema, that account needs DDL rights as well as ",[29,145,146],{},"SELECT",[127,148,149,152,153,155,156,159,160,163,164,111],{},[130,150,151],{},"vector",", on every host, writes events and nothing else. It never reads\n",[29,154,135],{},". Its endpoint, user and password come from the admin console\nunder ",[130,157,158],{},"Settings",", and are compiled into the ",[29,161,162],{},"vector.yaml"," the control plane\npushes to each host. That is the HTTP interface, port ",[29,165,166],{},"8123",[10,168,169,170,172],{},"So the URL in the admin settings is not the control plane's own connection, and\nthe two are usually not even the same address: the settings one has to be\nreachable from every QEMU host, while ",[29,171,135],{}," only has to be reachable\nfrom the control container.",[10,174,175],{},"The writer account only ever inserts, so grant it only that:",[22,177,181],{"className":178,"code":179,"language":180,"meta":27,"style":27},"language-sql shiki shiki-themes github-light github-dark","CREATE USER vector_writer IDENTIFIED WITH sha256_password BY 'a-strong-password';\nGRANT INSERT ON dummie.suricata_events TO vector_writer;\nGRANT INSERT ON dummie.dns_queries     TO vector_writer;\n","sql",[29,182,183,188,193],{"__ignoreMap":27},[32,184,185],{"class":34,"line":35},[32,186,187],{},"CREATE USER vector_writer IDENTIFIED WITH sha256_password BY 'a-strong-password';\n",[32,189,190],{"class":34,"line":49},[32,191,192],{},"GRANT INSERT ON dummie.suricata_events TO vector_writer;\n",[32,194,195],{"class":34,"line":59},[32,196,197],{},"GRANT INSERT ON dummie.dns_queries     TO vector_writer;\n",[10,199,200,201,204,205,208],{},"Then put ",[29,202,203],{},"vector_writer"," and its password into the ClickHouse user and password\nsettings in the console, and the ClickHouse URL as the hosts reach it —\n",[29,206,207],{},"http:\u002F\u002F10.0.0.1:8123",", say. Leaving the URL empty stops vector being installed\nat all, which is how you turn event shipping off.",[10,210,211],{},"Three notes on that grant:",[124,213,214,232,253],{},[127,215,216,221,222,224,225,228,229,231],{},[130,217,218,220],{},[29,219,146],{}," is not needed, and should not be given."," ",[29,223,162],{}," lands on\nevery host in the fleet, ",[29,226,227],{},"0600",", read by a root process. The credential in it\nis only as private as the least private machine you run. An insert-only account\ncaps a host compromise at forging events; one with ",[29,230,146],{}," hands over every\nguest's traffic history, and one with DDL hands over the tables.",[127,233,234,240,241,244,245,248,249,252],{},[130,235,236,237,111],{},"Don't grant on ",[29,238,239],{},"dummie.*"," Those two tables are the whole write path.\n",[29,242,243],{},"domain"," on ",[29,246,247],{},"suricata_events"," is a ",[29,250,251],{},"MATERIALIZED"," column the server computes,\nso vector never sends it and needs nothing extra for it.",[127,254,255,258,259,262,263,266,267,270],{},[130,256,257],{},"The account's profile must allow per-query settings."," vector sends\n",[29,260,261],{},"input_format_skip_unknown_fields"," on every insert, so a profile with\n",[29,264,265],{},"readonly = 1",", or with constraints on that setting, rejects the write even\nthough the grants are correct. The stock ",[29,268,269],{},"default"," profile is fine.",[10,272,273,274,277,278,281,282,285,286,288,289,292],{},"Running the ",[29,275,276],{},"CREATE USER"," and ",[29,279,280],{},"GRANT"," above needs an account with access\nmanagement enabled. On the official image that means ",[29,283,284],{},"CLICKHOUSE_ACCESS_MANAGEMENT=1","\nin ",[29,287,95],{},"; without it, define the second user in a\n",[29,290,291],{},"\u002Fetc\u002Fclickhouse-server\u002Fusers.d\u002F"," file instead.",[116,294,296],{"id":295},"behind-a-reverse-proxy","Behind a reverse proxy",[10,298,299,300,303],{},"Put the control plane on its own hostname — ",[29,301,302],{},"console.\u003Cyour-domain>"," is the\nconvention this project uses. Two things matter:",[124,305,306,319],{},[127,307,308,311,312,315,316,318],{},[130,309,310],{},"TLS."," The session cookie is issued ",[29,313,314],{},"Secure"," in production, and a browser\nwill discard a ",[29,317,314],{}," cookie that arrives over plain HTTP. Without TLS,\nsign-in appears to succeed and then does nothing.",[127,320,321,324],{},[130,322,323],{},"Host header."," Pass it through unmodified. Guest sessions are scoped by\nhostname, and rewriting it breaks the hand-off to a running VM.",[14,326,328],{"id":327},"hosts","Hosts",[10,330,331,332,335,336,338,339,92,342,345,346,351],{},"A host is a machine with nested virtualisation available, running ",[29,333,334],{},"dclient",".\n",[29,337,334],{}," registers with the control plane, then brings up the local services a\nsandbox needs — ",[29,340,341],{},"proxy",[29,343,344],{},"dpipe",", Suricata and CoreDNS — and manages their\nconfiguration as VMs come and go. See ",[347,348,350],"a",{"href":349},"\u002Fdocs\u002Farchitecture","Architecture"," for what\neach of those owns.",[10,353,354],{},"Hosts need:",[124,356,357,364,367],{},[127,358,359,360,363],{},"KVM available to the kernel (",[29,361,362],{},"\u002Fdev\u002Fkvm"," present and writable)",[127,365,366],{},"Docker, for the Suricata container",[127,368,369],{},"outbound reachability to the control plane",[116,371,373],{"id":372},"trying-it-without-hardware","Trying it without hardware",[10,375,376,377,379],{},"The repository carries a Nix flake that builds a VM with nested virtualisation\nenabled, which stands in for a real QEMU host. The control plane runs in Docker\non your machine, and ",[29,378,334],{}," runs inside the VM:",[22,381,383],{"className":24,"code":382,"language":26,"meta":27,"style":27},"just vm-start     # boot the stand-in host\njust vm-ssh       # get a shell on it\njust vm-stop      # tear it down\n",[29,384,385,397,407],{"__ignoreMap":27},[32,386,387,390,393],{"class":34,"line":35},[32,388,389],{"class":38},"just",[32,391,392],{"class":42}," vm-start",[32,394,396],{"class":395},"sJ8bj","     # boot the stand-in host\n",[32,398,399,401,404],{"class":34,"line":49},[32,400,389],{"class":38},[32,402,403],{"class":42}," vm-ssh",[32,405,406],{"class":395},"       # get a shell on it\n",[32,408,409,411,414],{"class":34,"line":59},[32,410,389],{"class":38},[32,412,413],{"class":42}," vm-stop",[32,415,416],{"class":395},"      # tear it down\n",[10,418,419],{},"This is the same code path as a production host, so it is a genuine test of a\nchange rather than a simulation of one.",[14,421,423],{"id":422},"configuring-the-marketing-site","Configuring the marketing site",[10,425,426,427,430],{},"If you are hosting the whole project — this site included — the website image\ntakes a single variable, ",[29,428,429],{},"CONSOLE_URL",", pointing at wherever your control plane\nlives. It is applied to the built site when the container starts, so the same\nimage serves any deployment and never needs rebuilding to be re-pointed:",[22,432,434],{"className":24,"code":433,"language":26,"meta":27,"style":27},"docker run -e CONSOLE_URL=https:\u002F\u002Fconsole.example.com -p 8080:8080 \\\n  codingcoffee\u002Fdummie-website\n",[29,435,436,458],{"__ignoreMap":27},[32,437,438,440,443,446,449,452,455],{"class":34,"line":35},[32,439,62],{"class":38},[32,441,442],{"class":42}," run",[32,444,445],{"class":52}," -e",[32,447,448],{"class":42}," CONSOLE_URL=https:\u002F\u002Fconsole.example.com",[32,450,451],{"class":52}," -p",[32,453,454],{"class":42}," 8080:8080",[32,456,457],{"class":52}," \\\n",[32,459,460],{"class":34,"line":49},[32,461,462],{"class":42},"  codingcoffee\u002Fdummie-website\n",[464,465,466],"style",{},"html pre.shiki code .sScJk, html code.shiki .sScJk{--shiki-default:#6F42C1;--shiki-dark:#B392F0}html pre.shiki code .sZZnC, html code.shiki .sZZnC{--shiki-default:#032F62;--shiki-dark:#9ECBFF}html pre.shiki code .sj4cs, html code.shiki .sj4cs{--shiki-default:#005CC5;--shiki-dark:#79B8FF}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html pre.shiki code .sJ8bj, html code.shiki .sJ8bj{--shiki-default:#6A737D;--shiki-dark:#6A737D}",{"title":27,"searchDepth":49,"depth":49,"links":468},[469,473,476],{"id":16,"depth":49,"text":17,"children":470},[471,472],{"id":118,"depth":59,"text":119},{"id":295,"depth":59,"text":296},{"id":327,"depth":49,"text":328,"children":474},[475],{"id":372,"depth":59,"text":373},{"id":422,"depth":49,"text":423},"Stand up the control plane with Docker Compose, then attach a host that can boot microVMs.","md",{},true,"\u002Fdocs\u002Fself-hosting","Getting started",{"title":5,"description":477},"docs\u002F02.self-hosting","cbdK-Aqd72T8EhxTLmwBzITJNIf_Y4bSd03hIWSpJGQ",[487,490],{"path":349,"title":350,"section":488,"stem":489},"Concepts","docs\u002F01.architecture",{"path":481,"title":5,"section":482,"stem":484},1787592506949]