Compare commits

...

82 Commits

Author SHA1 Message Date
Mauricio Siu
1379b2118f Merge branch 'canary' into feat/add-concurrent-builds 2025-11-17 22:05:31 -06:00
Mauricio Siu
8a741e41bb Merge pull request #2933 from AathilFelix/feat/server-time-clock
feat: add server time clock in the dashboard
2025-11-16 23:01:04 -06:00
Mauricio Siu
1581defc39 feat: conditionally render TimeBadge based on cloud status
- Updated the ShowProjects and side layout components to only display the TimeBadge when not in cloud mode.
- Modified the TimeBadge component to remove the refetch interval for server time when in cloud mode, returning null instead.
- Enhanced the server API to return null for server time in cloud environments, improving performance and avoiding unnecessary queries.
2025-11-16 21:32:23 -06:00
Mauricio Siu
f5891b8793 Merge branch 'canary' into feat/server-time-clock 2025-11-16 21:27:29 -06:00
Mauricio Siu
19244a2dea Merge pull request #3034 from Dokploy/3012-bug-v0256-database-restore-from-s3-gui-fails-due-to-unescaped-special-characters-in-the-database-password
fix: update database restore commands to properly quote user credentials
2025-11-16 15:45:18 -06:00
Mauricio Siu
c4c1930195 fix: update database restore commands to properly quote user credentials
- Modified the restore command functions for PostgreSQL, MariaDB, MySQL, and MongoDB to ensure that database user credentials are enclosed in single quotes. This change enhances command execution reliability and prevents potential issues with special characters in usernames and passwords.
2025-11-16 15:43:46 -06:00
Mauricio Siu
201cc65b09 Merge pull request #3027 from Dokploy/1925-remote-server-visibility-per-applicationservice
feat: enhance environment service to include server details
2025-11-15 23:55:11 -06:00
Mauricio Siu
3618be65fc feat: add server icon display in environment service dashboard
- Introduced a server icon next to services in the environment dashboard for better visual identification of server associations.
- Enhanced user experience by providing immediate visual cues regarding the server linked to each service.
2025-11-15 23:54:53 -06:00
Mauricio Siu
e9b4245625 feat: enhance environment service to include server details
- Added server information retrieval for applications and various database services in the environment service.
- Updated the dashboard to display server names alongside services, allowing for better identification and filtering of services by server.
- Introduced a dropdown filter for selecting services based on server, improving user experience in managing environments.
2025-11-15 23:51:34 -06:00
Mauricio Siu
e60c68dbeb Merge pull request #2989 from Harikrishnan1367709/Better-deployment-logs-for-long-commit-message-#2973
feat: Add expandable commit messages for deployment logs
2025-11-15 17:47:16 -06:00
Mauricio Siu
f46444e039 refactor: simplify deployment description toggle logic
- Removed the separate toggleDescription function and integrated its logic directly into the button's onClick handler for better readability.
- Maintained functionality for expanding and collapsing deployment descriptions while streamlining the code structure.
2025-11-15 17:46:14 -06:00
Mauricio Siu
05e3d241f1 feat: increase commit message truncation length and simplify truncation logic
- Updated the maximum character length for commit message truncation from 150 to 200 characters.
- Simplified the truncation logic by removing unnecessary checks and consolidating the function to focus solely on the new maximum length.
- Enhanced the display logic for deployment titles to ensure better readability and user experience.
2025-11-15 17:43:51 -06:00
Mauricio Siu
5c2bae2f21 Merge branch 'canary' into Better-deployment-logs-for-long-commit-message-#2973 2025-11-15 17:34:48 -06:00
Mauricio Siu
d854979fe3 Merge pull request #2984 from ChillerDragon/pr_fix_template_checkboxes
fix: pr template checkboxes
2025-11-15 17:34:20 -06:00
Mauricio Siu
794cd79973 feat: add comprehensive testing for grouped queue and queue manager functionality
- Introduced tests for the GroupedQueue class, covering basic functionality, concurrency handling, and job processing across multiple groups.
- Added tests for the QueueManager class, ensuring correct queue creation, job management, and handler functionality.
- Implemented tests for concurrency changes and their effects on pending tasks, enhancing overall test coverage for the queue system.
- Created a new ChangeConcurrencyModal component for adjusting deployment concurrency settings in the UI.
2025-11-15 17:31:52 -06:00
Mauricio Siu
8016708798 Merge pull request #3021 from Dokploy/1735-bug-in-server-monitoring
1735 bug in server monitoring
2025-11-15 01:00:40 -06:00
Mauricio Siu
09a98a29e0 fix: remove unnecessary console log from Docker stats monitoring 2025-11-15 00:59:36 -06:00
Mauricio Siu
a4caa47e10 feat: implement host system stats retrieval for Docker monitoring
- Added a new function `getHostSystemStats` to encapsulate the logic for retrieving host system statistics using `node-os-utils`.
- Refactored Docker stats monitoring to utilize the new function, improving code clarity and maintainability.
- Removed redundant OSUtils instantiation from the Docker stats monitoring logic.
2025-11-15 00:59:00 -06:00
Mauricio Siu
969147cd59 feat: enhance Docker stats monitoring with disk I/O statistics
- Updated OSUtils instantiation to include disk I/O statistics.
- Implemented filtering to exclude virtual devices from disk stats, ensuring only real disk devices are monitored.
2025-11-15 00:56:05 -06:00
Mauricio Siu
6369012389 Merge pull request #3020 from Dokploy/1735-bug-in-server-monitoring
chore: update node-os-utils to version 2.0.1 and refactor lodash imports
2025-11-15 00:30:39 -06:00
Mauricio Siu
69b7777db4 chore: update node-os-utils to version 2.0.1 and refactor lodash imports
- Upgraded `node-os-utils` from version 1.3.7 to 2.0.1 across multiple package.json files.
- Removed deprecated `@types/node-os-utils` dependency.
- Refactored lodash imports to use a single import statement for consistency.
- Enhanced Docker stats monitoring by integrating new features from `node-os-utils` version 2.0.1.
2025-11-15 00:28:44 -06:00
Mauricio Siu
b9324e6320 Merge pull request #3019 from Dokploy/fix/clean-railpack-builder-after-build
fix: ensure proper cleanup of Docker buildx builder container
2025-11-14 23:10:12 -06:00
Mauricio Siu
04a1a84077 fix: ensure proper cleanup of Docker buildx builder container
- Added commands to remove the builder container after Railpack build and prepare failures to prevent resource leaks.
- Improved bash command structure for better readability and maintenance.
2025-11-14 23:09:02 -06:00
Mauricio Siu
735b70b7fe Merge pull request #3018 from Dokploy/2508-git-based-deployments-should-have-git-hash-and-commit-message-on-deploy-manually
feat: add git commit info extraction to deployment logic
2025-11-14 22:31:36 -06:00
Mauricio Siu
61d9ae397a feat: add git commit info extraction to deployment logic
- Integrated `getGitCommitInfo` function to retrieve the latest commit message and hash for applications and compose services.
- Updated deployment logic to conditionally include commit information in deployment updates, enhancing traceability.
- Refactored import statements for better organization and clarity.
2025-11-14 22:27:38 -06:00
Mauricio Siu
ea5d86e295 Merge pull request #3000 from Bima42/fix/bump-traefik-version
chore: bump traefik to 3.6.1
2025-11-14 22:12:30 -06:00
Mauricio Siu
dd06c7006d Merge pull request #2513 from divaltor/docker-image-tag
feat(tags): Add support for tags from Github Packages
2025-11-14 01:35:39 -06:00
Mauricio Siu
4d36741e50 refactor: streamline service extraction logic in add-permissions component
- Updated type definitions for Environment and Project to improve clarity and maintainability.
- Refactored the extractServices function to use optional chaining and nullish coalescing for safer data handling.
- Enhanced type safety by casting the mapped services to the Services type.
2025-11-14 01:33:07 -06:00
Mauricio Siu
a9b9dd4b66 fix: conditionally include deployment hash in job data logging 2025-11-14 01:14:35 -06:00
Mauricio Siu
fbb1f1f266 fix: remove unnecessary log statement in Docker deploy validation 2025-11-14 01:11:52 -06:00
Mauricio Siu
c35fe0d457 feat: enhance Docker image handling in deployment logic
- Added functions to extract image name and tag from Docker images and webhook requests.
- Implemented validation for Docker image names and tags during deployment.
- Expanded test coverage for image tag extraction and commit message generation for GitHub Packages events.
- Improved error handling for missing image names and tags in deployment requests.
2025-11-14 01:10:49 -06:00
Mauricio Siu
ec081b6f2e Merge branch 'canary' into docker-image-tag 2025-11-13 22:55:55 -06:00
Mauricio Siu
4518ea2092 Merge pull request #3010 from Dokploy/Improve-project-view-by-showing-last-deploy-rather-than-created
feat: add last deployment date to services and update sorting logic
2025-11-13 22:36:48 -06:00
Mauricio Siu
d549aa6a62 feat: add last deployment date to services and update sorting logic
- Introduced `lastDeployDate` property to track the most recent deployment for applications and compose services.
- Updated the `extractServicesFromEnvironment` function to calculate and include the last deployment date.
- Modified sorting logic to allow sorting by last deployment date, enhancing the user experience on the environment dashboard.
- Adjusted local storage default sort preference to prioritize last deployment date.
2025-11-13 22:35:16 -06:00
Mauricio Siu
62474c1222 Merge pull request #2978 from Dokploy/unify-deployment-logic
Refactor compose and deployment services: streamline cloning and exec…
2025-11-13 22:25:50 -06:00
Mauricio Siu
26ff4075df Merge branch 'canary' into unify-deployment-logic 2025-11-13 21:06:00 -06:00
Mauricio Siu
22f704dd59 Merge pull request #2988 from Harikrishnan1367709/Invitation-Link-broken-on-Mac-#2986
fix: Add protocol prefix to invitation links (#2986)
2025-11-13 12:35:52 -06:00
Bima42
d22aa0583c chore: bump traefik to 3.6.1 2025-11-13 16:17:21 +01:00
autofix-ci[bot]
70bb32c590 [autofix.ci] apply automated fixes 2025-11-11 07:42:12 +00:00
HarikrishnanD
843313ddb9 feat: add expandable commit messages for deployment logs 2025-11-11 13:10:47 +05:30
HarikrishnanD
b202974a7d fix: add protocol prefix to invitation links 2025-11-11 11:34:10 +05:30
ChillerDragon
c56ddf3ec1 fix: pr template checkboxes
without a space they do not render as checkboxes on github
2025-11-10 11:12:00 +01:00
Mauricio Siu
b814bdc612 Refactor application and compose deployment logic: remove unused buildApplication function, streamline command logging for deployment, and enhance static command generation for improved maintainability and clarity in the codebase. 2025-11-09 11:13:39 -06:00
Mauricio Siu
d8ab7a59ff Refactor Bitbucket header utility: remove unused BitbucketClone type definition to streamline the code and enhance maintainability. 2025-11-09 03:43:54 -06:00
Mauricio Siu
f718ab334e Refactor compose utilities: remove unused functions and streamline the buildCompose logic for improved maintainability. Update domain handling by retaining only the necessary remote function, enhancing clarity in the codebase. 2025-11-09 03:42:43 -06:00
Mauricio Siu
668aaf9a91 Refactor deployment utilities: rename remote deployment functions for clarity and consistency, enhancing the deployment logic in the application. Streamline the build application function by commenting out unused build types to improve maintainability. 2025-11-09 03:29:40 -06:00
Mauricio Siu
ef10996dd8 Refactor builder utilities: remove unused build functions for Docker, Heroku, Nixpacks, Paketo, and Railpack, streamlining the codebase. Update static command generation to enhance clarity and maintainability. 2025-11-09 03:28:32 -06:00
Mauricio Siu
a05b75fc67 Refactor deployment logic: remove unused remote preview deployment function, streamline deployment commands, and enhance error handling for Docker image pulling. Update build command generation for Docker source type. 2025-11-09 03:24:13 -06:00
Mauricio Siu
f96114ad80 Refactor Bitbucket repository cloning logic: remove unused parameter and enhance error handling by retrieving Bitbucket provider directly within the function. 2025-11-09 03:18:07 -06:00
Mauricio Siu
5ac32f9f24 Refactor repository cloning interfaces: standardize parameters for Bitbucket, Gitea, and GitLab repository cloning functions to improve consistency and maintainability across the codebase. 2025-11-09 03:16:18 -06:00
Mauricio Siu
7b398939f7 Refactor compose and deployment services: streamline cloning and execution commands, remove redundant remote functions, and enhance error handling. Update database schema to include application build server ID for better tracking of deployments. 2025-11-09 03:12:49 -06:00
Mauricio Siu
fd8f0e8f1f Merge pull request #2950 from Bima42/fix/2949-upload-in-dropzone-two-times-in-a-row
fix: clear input value after uploading file in dropzone
2025-11-08 14:20:46 -06:00
Mauricio Siu
4f2268e66f Merge pull request #2976 from Dokploy/2838-endpoint-mode-configuration-solves-networking-issues-inside-lxc-containers
Refactor user schema and update database references: rename 'users_te…
2025-11-08 14:13:40 -06:00
Mauricio Siu
b99d532582 Update tests and refactor user query: Add 'endpointSpecSwarm' to application test cases and rename user variable in Stripe webhook to improve clarity and consistency. 2025-11-08 14:12:01 -06:00
Mauricio Siu
fb2bb99a2c Add SQL migration for user table refactor and endpoint specifications: Rename 'user_temp' to 'user', drop and add foreign key constraints, and introduce 'endpointSpecSwarm' column in multiple tables. Update journal and snapshot metadata for migration 0120_lame_captain_midlands. 2025-11-08 14:09:26 -06:00
Mauricio Siu
785172fa7b Enhance application schema and database utilities: Add 'endpointSpecSwarm' to application schema, update Docker container configuration to handle 'EndpointSpec' more flexibly across various database implementations, and remove deprecated 'generateEndpointSpec' function to streamline codebase. 2025-11-08 14:08:14 -06:00
Mauricio Siu
43701915f1 Add SQL migration for user table refactor: rename 'users' to 'user', update foreign key constraints, and add unique email constraint. Update journal and snapshot metadata for migration 0121_colorful_star_brand. 2025-11-08 13:57:05 -06:00
Mauricio Siu
2619733915 Refactor user schema and update database references: rename 'users_temp' to 'user' across the codebase, update related database queries, and enhance endpoint specifications for swarm settings in various database schemas. 2025-11-08 13:54:32 -06:00
Bima42
8aa496b773 fix: clear input value after uploading file in dropzone 2025-11-03 19:03:19 +01:00
Mauricio Siu
1ce153371a Merge pull request #2930 from Harikrishnan1367709/Add-the-ability-to-mark-an-organization-as--default--or-remember-last-used-organization-#1991
feat: Add ability to mark organization as default (#1991)
2025-11-02 22:33:38 -06:00
Mauricio Siu
41849654a7 Refactor Sidebar organization actions: streamline default organization setting and deletion logic, ensuring proper error handling and UI consistency. 2025-11-02 22:32:28 -06:00
Mauricio Siu
a475361b80 Refactor organization management in Sidebar: streamline organization selection and default setting logic. Update API to return organization memberships with default status. Improve UI for organization actions in the sidebar. 2025-11-02 22:27:04 -06:00
Mauricio Siu
1dc5bbd9bd Add 'is_default' column to 'member' table and update journal and snapshot metadata for migration 0119_bouncy_morbius 2025-11-02 22:07:20 -06:00
Mauricio Siu
d55e934978 Remove deprecated SQL migration file '0120_plain_eternity.sql' and corresponding entries from journal and snapshot metadata to clean up project structure. 2025-11-02 22:05:38 -06:00
Mauricio Siu
dddb866233 Remove 'is_default' field from snapshot metadata in 0114_snapshot.json to streamline project permissions configuration. 2025-11-02 22:04:05 -06:00
Mauricio Siu
0b58092c8a Remove deprecated SQL migration file and add new migration for default member organization flag. Update journal and snapshot metadata accordingly. 2025-11-02 22:03:34 -06:00
Mauricio Siu
759955e05e Delete apps/dokploy/drizzle/0114_sudden_sheva_callister.sql 2025-11-02 22:02:09 -06:00
Mauricio Siu
5949005458 Remove deprecated SQL migration file and add new migration for default member organization flag. Update journal and snapshot metadata accordingly. 2025-11-02 21:57:43 -06:00
Mauricio Siu
71b550f7e6 Merge branch 'canary' into Add-the-ability-to-mark-an-organization-as--default--or-remember-last-used-organization-#1991 2025-11-02 21:41:08 -06:00
Mauricio Siu
832a98734a Merge pull request #2943 from Dokploy/2905-subdomain-length-of-random-traefik-domain-isnt-checked-and-exceeds-maximum
feat(domain): truncate project name to comply with domain label lengt…
2025-11-02 17:21:41 -06:00
Mauricio Siu
65b3ce831f feat(domain): truncate project name to comply with domain label length restrictions 2025-11-02 17:20:42 -06:00
Mauricio Siu
6613cb7587 Merge pull request #2921 from Harikrishnan1367709/Container-not-started-if-the-Volume-contains-spaces-#2916
feat(volumes): block spaces/quotes in volume names (#2916)
2025-11-02 17:16:13 -06:00
Mauricio Siu
75a43896a2 Merge pull request #2941 from AtilMohAmine/fix/empty-json-responses-compose
Fix: Add JSON responses to compose endpoints that return empty body
2025-11-02 17:14:31 -06:00
test
64e48a7bbe fix: add JSON responses to compose endpoints that return empty body 2025-11-02 18:16:08 +01:00
Mauricio Siu
5434d9730d Merge pull request #2937 from amirparsadd/amirparsadd-patch-1
ArvanCloud new IP Ranges
2025-11-01 22:41:08 -06:00
Amirparsa Baghdadi
373c78a927 ArvanCloud new IP Ranges 2025-11-01 23:10:58 +03:30
Aathil Felix
53b66e41e2 chore(ui): apply Biome format to time badge and headers 2025-11-01 19:09:58 +05:30
Aathil Felix
0f100c7bc8 feat: add server time clock 2025-11-01 18:03:40 +05:30
autofix-ci[bot]
856b6ceec6 [autofix.ci] apply automated fixes 2025-10-31 14:53:42 +00:00
HarikrishnanD
a14cc09933 feat: Add default organization selection (#1991) 2025-10-31 20:21:49 +05:30
HarikrishnanD
94c00312c1 feat(volumes): reject spaces/quotes in volume names per Docker rules (#2916) 2025-10-30 12:54:37 +05:30
Vlad Vladov
6da122eab7 feat(tags): Add support for tags from Github Packages 2025-09-03 18:05:22 +03:00
103 changed files with 17816 additions and 3308 deletions

View File

@@ -6,9 +6,9 @@ Please describe in a short paragraph what this PR is about.
Before submitting this PR, please make sure that:
- [] You created a dedicated branch based on the `canary` branch.
- [] You have read the suggestions in the CONTRIBUTING.md file https://github.com/Dokploy/dokploy/blob/canary/CONTRIBUTING.md#pull-request
- [] You have tested this PR in your local instance.
- [ ] You created a dedicated branch based on the `canary` branch.
- [ ] You have read the suggestions in the CONTRIBUTING.md file https://github.com/Dokploy/dokploy/blob/canary/CONTRIBUTING.md#pull-request
- [ ] You have tested this PR in your local instance.
## Issues related (if applicable)

View File

@@ -1,9 +1,9 @@
import {
deployRemoteApplication,
deployRemoteCompose,
deployRemotePreviewApplication,
rebuildRemoteApplication,
rebuildRemoteCompose,
deployApplication,
deployCompose,
deployPreviewApplication,
rebuildApplication,
rebuildCompose,
updateApplicationStatus,
updateCompose,
updatePreviewDeployment,
@@ -16,13 +16,13 @@ export const deploy = async (job: DeployJob) => {
await updateApplicationStatus(job.applicationId, "running");
if (job.server) {
if (job.type === "redeploy") {
await rebuildRemoteApplication({
await rebuildApplication({
applicationId: job.applicationId,
titleLog: job.titleLog || "Rebuild deployment",
descriptionLog: job.descriptionLog || "",
});
} else if (job.type === "deploy") {
await deployRemoteApplication({
await deployApplication({
applicationId: job.applicationId,
titleLog: job.titleLog || "Manual deployment",
descriptionLog: job.descriptionLog || "",
@@ -36,13 +36,13 @@ export const deploy = async (job: DeployJob) => {
if (job.server) {
if (job.type === "redeploy") {
await rebuildRemoteCompose({
await rebuildCompose({
composeId: job.composeId,
titleLog: job.titleLog || "Rebuild deployment",
descriptionLog: job.descriptionLog || "",
});
} else if (job.type === "deploy") {
await deployRemoteCompose({
await deployCompose({
composeId: job.composeId,
titleLog: job.titleLog || "Manual deployment",
descriptionLog: job.descriptionLog || "",
@@ -55,7 +55,7 @@ export const deploy = async (job: DeployJob) => {
});
if (job.server) {
if (job.type === "deploy") {
await deployRemotePreviewApplication({
await deployPreviewApplication({
applicationId: job.applicationId,
titleLog: job.titleLog || "Preview Deployment",
descriptionLog: job.descriptionLog || "",

View File

@@ -1,5 +1,10 @@
import { describe, expect, it } from "vitest";
import { extractCommitMessage } from "@/pages/api/deploy/[refreshToken]";
import {
extractCommitMessage,
extractImageName,
extractImageTag,
extractImageTagFromRequest,
} from "@/pages/api/deploy/[refreshToken]";
describe("GitHub Webhook Skip CI", () => {
const mockGithubHeaders = {
@@ -96,3 +101,308 @@ describe("GitHub Webhook Skip CI", () => {
);
});
});
describe("GitHub Packages Docker Image Tag Extraction", () => {
it("should extract tag from container_metadata", () => {
const headers = { "x-github-event": "registry_package" };
const body = {
registry_package: {
package_version: {
version: "sha256:abc123...",
container_metadata: {
tag: {
name: "v1.0.0",
digest: "sha256:abc123...",
},
},
package_url: "ghcr.io/owner/repo:v1.0.0",
},
},
};
const tag = extractImageTagFromRequest(headers, body);
expect(tag).toBe("v1.0.0");
});
it("should extract tag from package_url when container_metadata tag matches version", () => {
const headers = { "x-github-event": "registry_package" };
const body = {
registry_package: {
package_version: {
version: "sha256:abc123...",
container_metadata: {
tag: {
name: "sha256:abc123...",
digest: "sha256:abc123...",
},
},
package_url: "ghcr.io/owner/repo:latest",
},
},
};
const tag = extractImageTagFromRequest(headers, body);
expect(tag).toBe("latest");
});
it("should extract tag from package_url when container_metadata is missing", () => {
const headers = { "x-github-event": "registry_package" };
const body = {
registry_package: {
package_version: {
version: "sha256:abc123...",
package_url: "ghcr.io/owner/repo:1.2.3",
},
},
};
const tag = extractImageTagFromRequest(headers, body);
expect(tag).toBe("1.2.3");
});
it("should handle different tag formats in package_url", () => {
const headers = { "x-github-event": "registry_package" };
const testCases = [
{ url: "ghcr.io/owner/repo:latest", expected: "latest" },
{ url: "ghcr.io/owner/repo:v1.0.0", expected: "v1.0.0" },
{ url: "ghcr.io/owner/repo:1.2.3", expected: "1.2.3" },
{ url: "ghcr.io/owner/repo:dev", expected: "dev" },
];
for (const testCase of testCases) {
const body = {
registry_package: {
package_version: {
version: "sha256:abc123...",
package_url: testCase.url,
},
},
};
const tag = extractImageTagFromRequest(headers, body);
expect(tag).toBe(testCase.expected);
}
});
it("should return null for non-registry_package events", () => {
const headers = { "x-github-event": "push" };
const body = {
registry_package: {
package_version: {
package_url: "ghcr.io/owner/repo:latest",
},
},
};
const tag = extractImageTagFromRequest(headers, body);
expect(tag).toBeNull();
});
it("should return null when package_version is missing", () => {
const headers = { "x-github-event": "registry_package" };
const body = {
registry_package: {},
};
const tag = extractImageTagFromRequest(headers, body);
expect(tag).toBeNull();
});
it("should return null when package_url has no tag", () => {
const headers = { "x-github-event": "registry_package" };
const body = {
registry_package: {
package_version: {
version: "sha256:abc123...",
package_url: "ghcr.io/owner/repo",
},
},
};
const tag = extractImageTagFromRequest(headers, body);
expect(tag).toBeNull();
});
it("should return null when package_url ends with colon (no tag)", () => {
const headers = { "x-github-event": "registry_package" };
const body = {
registry_package: {
package_version: {
version: "sha256:abc123...",
package_url: "ghcr.io/owner/repo:",
container_metadata: {
tag: {
name: "",
digest: "sha256:abc123...",
},
},
},
},
};
const tag = extractImageTagFromRequest(headers, body);
expect(tag).toBeNull();
});
it("should return null when tag name is empty string", () => {
const headers = { "x-github-event": "registry_package" };
const body = {
registry_package: {
package_version: {
version: "sha256:abc123...",
container_metadata: {
tag: {
name: "",
digest: "sha256:abc123...",
},
},
package_url: "ghcr.io/owner/repo:",
},
},
};
const tag = extractImageTagFromRequest(headers, body);
expect(tag).toBeNull();
});
it("should ignore tag if it matches the version (digest)", () => {
const headers = { "x-github-event": "registry_package" };
const body = {
registry_package: {
package_version: {
version: "sha256:abc123...",
container_metadata: {
tag: {
name: "sha256:abc123...",
digest: "sha256:abc123...",
},
},
package_url: "ghcr.io/owner/repo:latest",
},
},
};
const tag = extractImageTagFromRequest(headers, body);
expect(tag).toBe("latest");
});
it("should handle registry_package commit message with package_url", () => {
const headers = { "x-github-event": "registry_package" };
const body = {
registry_package: {
package_version: {
package_url: "ghcr.io/owner/repo:latest",
},
},
};
const message = extractCommitMessage(headers, body);
expect(message).toBe("Docker GHCR image pushed: ghcr.io/owner/repo:latest");
});
it("should handle registry_package commit message when package_url is missing", () => {
const headers = { "x-github-event": "registry_package" };
const body = {
registry_package: {
package_version: {
version: "sha256:abc123...",
},
},
};
const message = extractCommitMessage(headers, body);
expect(message).toBe("Docker GHCR image pushed");
});
it("should handle registry_package commit message when package_version is missing", () => {
const headers = { "x-github-event": "registry_package" };
const body = {
registry_package: {},
};
const message = extractCommitMessage(headers, body);
expect(message).toBe("NEW COMMIT");
});
});
describe("Docker Image Name and Tag Extraction", () => {
describe("extractImageName", () => {
it("should return image name without tag", () => {
expect(extractImageName("my-image:latest")).toBe("my-image");
expect(extractImageName("my-image:1.0.0")).toBe("my-image");
expect(extractImageName("ghcr.io/owner/repo:latest")).toBe(
"ghcr.io/owner/repo",
);
});
it("should return full image name when no tag is present", () => {
expect(extractImageName("my-image")).toBe("my-image");
expect(extractImageName("ghcr.io/owner/repo")).toBe("ghcr.io/owner/repo");
});
it("should handle images with port numbers correctly", () => {
expect(extractImageName("registry:5000/image:tag")).toBe(
"registry:5000/image",
);
expect(extractImageName("localhost:5000/my-app:latest")).toBe(
"localhost:5000/my-app",
);
});
it("should handle complex image paths", () => {
expect(
extractImageName("myregistryhost:5000/fedora/httpd:version1.0"),
).toBe("myregistryhost:5000/fedora/httpd");
expect(extractImageName("registry.example.com:8080/ns/app:v1.2.3")).toBe(
"registry.example.com:8080/ns/app",
);
});
it("should return null for invalid inputs", () => {
expect(extractImageName(null)).toBeNull();
expect(extractImageName("")).toBeNull();
});
it("should handle edge cases with multiple colons", () => {
expect(extractImageName("image:tag:extra")).toBe("image:tag");
expect(extractImageName("registry:5000:invalid")).toBe("registry:5000");
});
});
describe("extractImageTag", () => {
it("should extract tag from image with tag", () => {
expect(extractImageTag("my-image:latest")).toBe("latest");
expect(extractImageTag("my-image:1.0.0")).toBe("1.0.0");
expect(extractImageTag("ghcr.io/owner/repo:v1.2.3")).toBe("v1.2.3");
});
it("should return 'latest' when no tag is present", () => {
expect(extractImageTag("my-image")).toBe("latest");
expect(extractImageTag("ghcr.io/owner/repo")).toBe("latest");
});
it("should handle complex image paths with tags", () => {
expect(
extractImageTag("myregistryhost:5000/fedora/httpd:version1.0"),
).toBe("version1.0");
expect(extractImageTag("registry.example.com:8080/ns/app:v1.2.3")).toBe(
"v1.2.3",
);
});
it("should return null for invalid inputs", () => {
expect(extractImageTag(null)).toBeNull();
expect(extractImageTag("")).toBeNull();
});
it("should handle edge cases with multiple colons", () => {
expect(extractImageTag("image:tag:extra")).toBe("extra");
expect(extractImageTag("registry:5000/image:tag")).toBe("tag");
});
it("should handle numeric tags", () => {
expect(extractImageTag("my-image:123")).toBe("123");
expect(extractImageTag("my-image:1")).toBe("1");
});
});
});

View File

@@ -42,6 +42,7 @@ const baseApp: ApplicationNested = {
triggerType: "push",
appName: "",
autoDeploy: true,
endpointSpecSwarm: null,
serverId: "",
registryUrl: "",
branch: null,

View File

@@ -0,0 +1,809 @@
import { describe, expect, it } from "vitest";
import { GroupedQueue } from "../../server/queues/grouped-queue-wrapper";
describe("GroupedQueue", () => {
describe("Basic functionality", () => {
it("should process a single job with concurrency 1", async () => {
const queue = new GroupedQueue<{ id: string }>(1);
const processed: string[] = [];
queue.setHandler(async (data) => {
await new Promise((resolve) => setTimeout(resolve, 50));
processed.push(data.id);
});
await queue.add("group1", { id: "job1" });
// Wait for processing to complete
await new Promise((resolve) => setTimeout(resolve, 100));
expect(processed).toEqual(["job1"]);
expect(queue.isIdle()).toBe(true);
});
it("should process jobs in FIFO order within a group", async () => {
const queue = new GroupedQueue<{ id: string }>(1);
const processed: string[] = [];
queue.setHandler(async (data) => {
await new Promise((resolve) => setTimeout(resolve, 20));
processed.push(data.id);
});
// Add multiple jobs to the same group
await Promise.all([
queue.add("group1", { id: "job1" }),
queue.add("group1", { id: "job2" }),
queue.add("group1", { id: "job3" }),
]);
// Wait for all processing
await new Promise((resolve) => setTimeout(resolve, 200));
expect(processed).toEqual(["job1", "job2", "job3"]);
});
});
describe("Concurrency 1 with multiple groups", () => {
it("should process one group at a time with concurrency 1", async () => {
const queue = new GroupedQueue<{ id: string; group: string }>(1);
const processed: string[] = [];
const activeGroups: string[] = [];
queue.setHandler(async (data) => {
activeGroups.push(data.group);
await new Promise((resolve) => setTimeout(resolve, 50));
processed.push(data.id);
activeGroups.pop();
});
// Add jobs to 3 different groups
const promises = [
queue.add("app1", { id: "job1", group: "app1" }),
queue.add("app2", { id: "job2", group: "app2" }),
queue.add("app3", { id: "job3", group: "app3" }),
];
// Check after 30ms - only one should be processing
await new Promise((resolve) => setTimeout(resolve, 30));
expect(activeGroups.length).toBeLessThanOrEqual(1);
// Wait for all to complete
await Promise.all(promises);
await new Promise((resolve) => setTimeout(resolve, 200));
expect(processed).toHaveLength(3);
expect(queue.isIdle()).toBe(true);
});
it("should process groups sequentially with concurrency 1", async () => {
const queue = new GroupedQueue<{ id: string; group: string }>(1);
const processingOrder: string[] = [];
const startTimes: Map<string, number> = new Map();
const endTimes: Map<string, number> = new Map();
queue.setHandler(async (data) => {
startTimes.set(data.id, Date.now());
processingOrder.push(`start-${data.group}`);
await new Promise((resolve) => setTimeout(resolve, 50));
endTimes.set(data.id, Date.now());
processingOrder.push(`end-${data.group}`);
});
await Promise.all([
queue.add("app1", { id: "job1", group: "app1" }),
queue.add("app2", { id: "job2", group: "app2" }),
queue.add("app3", { id: "job3", group: "app3" }),
]);
await new Promise((resolve) => setTimeout(resolve, 300));
// Verify sequential processing
expect(processingOrder).toEqual([
"start-app1",
"end-app1",
"start-app2",
"end-app2",
"start-app3",
"end-app3",
]);
// Verify jobs don't overlap
const job1End = endTimes.get("job1")!;
const job2Start = startTimes.get("job2")!;
const job2End = endTimes.get("job2")!;
const job3Start = startTimes.get("job3")!;
expect(job2Start).toBeGreaterThanOrEqual(job1End);
expect(job3Start).toBeGreaterThanOrEqual(job2End);
});
});
describe("Concurrency 3 with 4 groups", () => {
it("should process up to 3 groups simultaneously", async () => {
const queue = new GroupedQueue<{ id: string; group: string }>(3);
const activeGroups = new Set<string>();
const maxConcurrent = { value: 0 };
queue.setHandler(async (data) => {
activeGroups.add(data.group);
maxConcurrent.value = Math.max(maxConcurrent.value, activeGroups.size);
await new Promise((resolve) => setTimeout(resolve, 100));
activeGroups.delete(data.group);
});
// Add 4 jobs to different groups
await Promise.all([
queue.add("app1", { id: "job1", group: "app1" }),
queue.add("app2", { id: "job2", group: "app2" }),
queue.add("app3", { id: "job3", group: "app3" }),
queue.add("app4", { id: "job4", group: "app4" }),
]);
// Check during processing
await new Promise((resolve) => setTimeout(resolve, 50));
// Should have processed 3 groups simultaneously
expect(maxConcurrent.value).toBe(3);
expect(activeGroups.size).toBeLessThanOrEqual(3);
// Wait for all to complete
await new Promise((resolve) => setTimeout(resolve, 200));
expect(queue.isIdle()).toBe(true);
});
it("should process 4th group after one of the first 3 completes", async () => {
const queue = new GroupedQueue<{ id: string; group: string }>(3);
const processingOrder: string[] = [];
queue.setHandler(async (data) => {
processingOrder.push(`start-${data.group}`);
await new Promise((resolve) => setTimeout(resolve, 100));
processingOrder.push(`end-${data.group}`);
});
await Promise.all([
queue.add("app1", { id: "job1", group: "app1" }),
queue.add("app2", { id: "job2", group: "app2" }),
queue.add("app3", { id: "job3", group: "app3" }),
queue.add("app4", { id: "job4", group: "app4" }),
]);
await new Promise((resolve) => setTimeout(resolve, 250));
// First 3 should start together
const firstThree = processingOrder.slice(0, 3);
expect(firstThree).toContain("start-app1");
expect(firstThree).toContain("start-app2");
expect(firstThree).toContain("start-app3");
// 4th should start after one completes
const app4StartIndex = processingOrder.indexOf("start-app4");
expect(app4StartIndex).toBeGreaterThan(0);
expect(app4StartIndex).toBeLessThan(processingOrder.length - 1);
});
});
describe("Multiple jobs per group", () => {
it("should process jobs sequentially within same group", async () => {
const queue = new GroupedQueue<{ id: string }>(3);
const processed: string[] = [];
queue.setHandler(async (data) => {
await new Promise((resolve) => setTimeout(resolve, 30));
processed.push(data.id);
});
// Add 3 jobs to same group
await Promise.all([
queue.add("app1", { id: "job1" }),
queue.add("app1", { id: "job2" }),
queue.add("app1", { id: "job3" }),
]);
await new Promise((resolve) => setTimeout(resolve, 200));
// Should process in order
expect(processed).toEqual(["job1", "job2", "job3"]);
});
it("should process multiple groups with multiple jobs each", async () => {
const queue = new GroupedQueue<{ id: string; group: string }>(2);
const processed: string[] = [];
queue.setHandler(async (data) => {
await new Promise((resolve) => setTimeout(resolve, 20));
processed.push(`${data.group}-${data.id}`);
});
// Add jobs to 2 groups, 2 jobs each
await Promise.all([
queue.add("app1", { id: "job1", group: "app1" }),
queue.add("app1", { id: "job2", group: "app1" }),
queue.add("app2", { id: "job1", group: "app2" }),
queue.add("app2", { id: "job2", group: "app2" }),
]);
await new Promise((resolve) => setTimeout(resolve, 200));
// Should process both groups, jobs within each group in order
expect(processed).toHaveLength(4);
expect(processed.filter((p) => p.startsWith("app1"))).toEqual([
"app1-job1",
"app1-job2",
]);
expect(processed.filter((p) => p.startsWith("app2"))).toEqual([
"app2-job1",
"app2-job2",
]);
});
});
describe("Error handling", () => {
it("should reject job on handler error", async () => {
const queue = new GroupedQueue<{ id: string }>(1);
queue.setHandler(async () => {
throw new Error("Test error");
});
await expect(queue.add("group1", { id: "job1" })).rejects.toThrow(
"Test error",
);
});
it("should continue processing other jobs after error", async () => {
const queue = new GroupedQueue<{ id: string }>(1);
const processed: string[] = [];
queue.setHandler(async (data) => {
if (data.id === "job2") {
throw new Error("Job 2 error");
}
processed.push(data.id);
});
await expect(
queue.add("group1", { id: "job1" }),
).resolves.toBeUndefined();
await expect(queue.add("group1", { id: "job2" })).rejects.toThrow();
await expect(
queue.add("group1", { id: "job3" }),
).resolves.toBeUndefined();
await new Promise((resolve) => setTimeout(resolve, 100));
expect(processed).toEqual(["job1", "job3"]);
});
});
describe("Queue management", () => {
it("should clear group tasks", async () => {
const queue = new GroupedQueue<{ id: string }>(1);
const processed: string[] = [];
queue.setHandler(async (data) => {
await new Promise((resolve) => setTimeout(resolve, 50));
processed.push(data.id);
});
// Add jobs without awaiting - they'll start processing
const job1Promise = queue.add("group1", { id: "job1" });
const job2Promise = queue.add("group1", { id: "job2" });
// Clear immediately - job1 might be processing, but job2 should be cleared
queue.clearGroup("group1");
// Use Promise.allSettled to handle both promises properly
const results = await Promise.allSettled([job1Promise, job2Promise]);
// job1 might succeed or fail depending on timing
// job2 should be rejected
const job2Result = results[1];
if (job2Result.status === "rejected") {
expect(job2Result.reason.message).toBe("Queue cleared");
}
await new Promise((resolve) => setTimeout(resolve, 100));
// Job1 might have processed, but job2 should not
expect(processed.length).toBeLessThanOrEqual(1);
});
it("should return correct group length", async () => {
const queue = new GroupedQueue<{ id: string }>(1);
queue.setHandler(async () => {
await new Promise((resolve) => setTimeout(resolve, 100));
});
// Add jobs without awaiting - check length immediately
const promises = [
queue.add("group1", { id: "job1" }),
queue.add("group1", { id: "job2" }),
queue.add("group1", { id: "job3" }),
];
// Check length immediately - at least some should be pending
// (job1 might be processing, but job2 and job3 should be pending)
const length = queue.getGroupLength("group1");
expect(length).toBeGreaterThanOrEqual(0);
// Wait for all to complete
await Promise.all(promises);
await new Promise((resolve) => setTimeout(resolve, 50));
// After processing should be 0
expect(queue.getGroupLength("group1")).toBe(0);
});
it("should close queue and reject pending tasks", async () => {
const queue = new GroupedQueue<{ id: string }>(1);
queue.setHandler(async () => {
await new Promise((resolve) => setTimeout(resolve, 100));
});
// Add first job and wait a bit to ensure it starts processing
const job1Promise = queue.add("group1", { id: "job1" });
// Add second job without awaiting
const job2Promise = queue.add("group1", { id: "job2" });
// Wait a tiny bit to ensure job2 is queued
await new Promise((resolve) => setTimeout(resolve, 10));
// Close queue - job2 should be rejected
await queue.close();
// Use Promise.allSettled to handle both promises properly
const results = await Promise.allSettled([job1Promise, job2Promise]);
// job1 might succeed or fail depending on timing
// job2 should be rejected
const job2Result = results[1];
if (job2Result.status === "rejected") {
expect(job2Result.reason.message).toBe("Queue closed");
}
});
});
describe("Concurrency edge cases", () => {
it("should handle concurrency 1 with 1 app correctly", async () => {
const queue = new GroupedQueue<{ id: string }>(1);
const processed: string[] = [];
queue.setHandler(async (data) => {
await new Promise((resolve) => setTimeout(resolve, 50));
processed.push(data.id);
});
await queue.add("app1", { id: "job1" });
await new Promise((resolve) => setTimeout(resolve, 100));
expect(processed).toEqual(["job1"]);
expect(queue.getActiveGroupsCount()).toBe(0);
});
it("should handle concurrency 1 with 3 apps correctly", async () => {
const queue = new GroupedQueue<{ id: string; app: string }>(1);
const processingTimes: Map<string, { start: number; end: number }> =
new Map();
queue.setHandler(async (data) => {
const start = Date.now();
await new Promise((resolve) => setTimeout(resolve, 50));
const end = Date.now();
processingTimes.set(data.app, { start, end });
});
await Promise.all([
queue.add("app1", { id: "job1", app: "app1" }),
queue.add("app2", { id: "job2", app: "app2" }),
queue.add("app3", { id: "job3", app: "app3" }),
]);
await new Promise((resolve) => setTimeout(resolve, 300));
// Verify sequential processing
const app1 = processingTimes.get("app1")!;
const app2 = processingTimes.get("app2")!;
const app3 = processingTimes.get("app3")!;
expect(app2.start).toBeGreaterThanOrEqual(app1.end);
expect(app3.start).toBeGreaterThanOrEqual(app2.end);
expect(queue.getActiveGroupsCount()).toBe(0);
});
it("should handle 4 apps with concurrency 3 correctly", async () => {
const queue = new GroupedQueue<{ id: string; app: string }>(3);
const concurrentCounts: number[] = [];
queue.setHandler(async () => {
// Track concurrent processing
const interval = setInterval(() => {
concurrentCounts.push(queue.getActiveGroupsCount());
}, 10);
await new Promise((resolve) => setTimeout(resolve, 100));
clearInterval(interval);
});
await Promise.all([
queue.add("app1", { id: "job1", app: "app1" }),
queue.add("app2", { id: "job2", app: "app2" }),
queue.add("app3", { id: "job3", app: "app3" }),
queue.add("app4", { id: "job4", app: "app4" }),
]);
await new Promise((resolve) => setTimeout(resolve, 200));
// Should never exceed concurrency of 3
const maxConcurrent = Math.max(...concurrentCounts);
expect(maxConcurrent).toBeLessThanOrEqual(3);
expect(queue.getActiveGroupsCount()).toBe(0);
});
});
describe("Idle state", () => {
it("should be idle when no jobs are processing", () => {
const queue = new GroupedQueue<{ id: string }>(1);
expect(queue.isIdle()).toBe(true);
});
it("should not be idle while processing", async () => {
const queue = new GroupedQueue<{ id: string }>(1);
let isIdleDuringProcessing = false;
queue.setHandler(async () => {
isIdleDuringProcessing = queue.isIdle();
await new Promise((resolve) => setTimeout(resolve, 50));
});
await queue.add("group1", { id: "job1" });
await new Promise((resolve) => setTimeout(resolve, 30));
expect(isIdleDuringProcessing).toBe(false);
expect(queue.isIdle()).toBe(true);
});
});
describe("Concurrency management", () => {
it("should get current concurrency", () => {
const queue1 = new GroupedQueue<{ id: string }>(1);
const queue2 = new GroupedQueue<{ id: string }>(5);
const queue3 = new GroupedQueue<{ id: string }>(10);
expect(queue1.getConcurrency()).toBe(1);
expect(queue2.getConcurrency()).toBe(5);
expect(queue3.getConcurrency()).toBe(10);
});
it("should set concurrency dynamically", () => {
const queue = new GroupedQueue<{ id: string }>(1);
expect(queue.getConcurrency()).toBe(1);
queue.setConcurrency(3);
expect(queue.getConcurrency()).toBe(3);
queue.setConcurrency(5);
expect(queue.getConcurrency()).toBe(5);
});
it("should throw error when setting concurrency less than 1", () => {
const queue = new GroupedQueue<{ id: string }>(1);
expect(() => queue.setConcurrency(0)).toThrow(
"Concurrency must be at least 1",
);
expect(() => queue.setConcurrency(-1)).toThrow(
"Concurrency must be at least 1",
);
});
it("should process next group when concurrency increases", async () => {
const queue = new GroupedQueue<{ id: string }>(1);
const processed: string[] = [];
queue.setHandler(async (data) => {
await new Promise((resolve) => setTimeout(resolve, 50));
processed.push(data.id);
});
// Add jobs to 3 different groups with concurrency 1
const job1Promise = queue.add("group1", { id: "job1" });
const job2Promise = queue.add("group2", { id: "job2" });
const job3Promise = queue.add("group3", { id: "job3" });
// Wait a bit to ensure job1 starts processing
await new Promise((resolve) => setTimeout(resolve, 10));
// Increase concurrency to 3 - should allow group2 and group3 to start
queue.setConcurrency(3);
// Wait for all to complete
await Promise.all([job1Promise, job2Promise, job3Promise]);
await new Promise((resolve) => setTimeout(resolve, 100));
expect(processed).toHaveLength(3);
expect(processed).toContain("job1");
expect(processed).toContain("job2");
expect(processed).toContain("job3");
});
});
describe("Clear all pending tasks", () => {
it("should clear all pending tasks across all groups", async () => {
const queue = new GroupedQueue<{ id: string }>(1);
const processed: string[] = [];
queue.setHandler(async (data) => {
await new Promise((resolve) => setTimeout(resolve, 100));
processed.push(data.id);
});
// Add multiple jobs to different groups
const job1Promise = queue.add("group1", { id: "job1" });
const job2Promise = queue.add("group1", { id: "job2" });
const job3Promise = queue.add("group2", { id: "job3" });
const job4Promise = queue.add("group2", { id: "job4" });
const job5Promise = queue.add("group3", { id: "job5" });
// Wait a bit to ensure job1 starts processing
await new Promise((resolve) => setTimeout(resolve, 10));
// Clear all pending tasks
const clearedCount = queue.clearAllPendingTasks();
// Should have cleared 4 pending tasks (job2, job3, job4, job5)
// job1 is processing so it's not in the queue anymore
expect(clearedCount).toBe(4);
// Handle all promises
const results = await Promise.allSettled([
job1Promise,
job2Promise,
job3Promise,
job4Promise,
job5Promise,
]);
// job1 should succeed (it was processing)
const job1Result = results[0];
expect(job1Result.status).toBe("fulfilled");
// All pending jobs should be rejected
for (let i = 1; i < results.length; i++) {
const result = results[i];
if (result && result.status === "rejected") {
expect(result.reason.message).toBe(
"Concurrency changed - queue cleared",
);
}
}
// Wait for job1 to complete
await new Promise((resolve) => setTimeout(resolve, 150));
// Only job1 should have processed
expect(processed).toHaveLength(1);
expect(processed).toContain("job1");
});
it("should not clear tasks that are currently processing", async () => {
const queue = new GroupedQueue<{ id: string }>(1);
const processed: string[] = [];
queue.setHandler(async (data) => {
await new Promise((resolve) => setTimeout(resolve, 100));
processed.push(data.id);
});
// Add jobs - first one will start processing immediately
const job1Promise = queue.add("group1", { id: "job1" });
const job2Promise = queue.add("group1", { id: "job2" });
// Wait to ensure job1 is processing (it's been shifted from tasks)
await new Promise((resolve) => setTimeout(resolve, 20));
// Clear all pending - should only clear job2, not job1
// job1 is already executing (not in tasks array)
const clearedCount = queue.clearAllPendingTasks();
expect(clearedCount).toBe(1);
// Handle all promises
const results = await Promise.allSettled([job1Promise, job2Promise]);
// job1 should succeed (it was processing)
const job1Result = results[0];
expect(job1Result.status).toBe("fulfilled");
// job2 should be rejected
const job2Result = results[1];
if (job2Result && job2Result.status === "rejected") {
expect(job2Result.reason.message).toBe(
"Concurrency changed - queue cleared",
);
}
await new Promise((resolve) => setTimeout(resolve, 50));
// Only job1 should have processed
expect(processed).toHaveLength(1);
expect(processed).toContain("job1");
});
it("should return 0 when no pending tasks", () => {
const queue = new GroupedQueue<{ id: string }>(1);
const clearedCount = queue.clearAllPendingTasks();
expect(clearedCount).toBe(0);
});
it("should clear tasks from multiple groups", async () => {
const queue = new GroupedQueue<{ id: string }>(1);
const processed: string[] = [];
queue.setHandler(async (data) => {
await new Promise((resolve) => setTimeout(resolve, 50));
processed.push(data.id);
});
// Add jobs to multiple groups
const promises = [
queue.add("group1", { id: "job1" }),
queue.add("group1", { id: "job2" }),
queue.add("group2", { id: "job3" }),
queue.add("group2", { id: "job4" }),
queue.add("group3", { id: "job5" }),
];
// Wait a bit for first job to start (it gets shifted from tasks)
await new Promise((resolve) => setTimeout(resolve, 10));
// Clear all pending
const clearedCount = queue.clearAllPendingTasks();
// Should clear 4 tasks (job2, job3, job4, job5)
// job1 is processing so it's not in the queue anymore
expect(clearedCount).toBe(4);
// Handle all promises
const results = await Promise.allSettled(promises);
// job1 should succeed
const job1Result = results[0];
expect(job1Result?.status).toBe("fulfilled");
// Others should be rejected
for (let i = 1; i < results.length; i++) {
const result = results[i];
if (result && result.status === "rejected") {
expect(result.reason.message).toBe(
"Concurrency changed - queue cleared",
);
}
}
await new Promise((resolve) => setTimeout(resolve, 100));
// Only first job should process
expect(processed.length).toBeLessThanOrEqual(1);
});
});
describe("Concurrency change with pending tasks", () => {
it("should clear pending tasks when concurrency changes", async () => {
const queue = new GroupedQueue<{ id: string }>(1);
const processed: string[] = [];
queue.setHandler(async (data) => {
await new Promise((resolve) => setTimeout(resolve, 50));
processed.push(data.id);
});
// Add jobs with concurrency 1
const job1Promise = queue.add("group1", { id: "job1" });
const job2Promise = queue.add("group1", { id: "job2" });
const job3Promise = queue.add("group2", { id: "job3" });
// Wait for job1 to start processing (it gets shifted from tasks)
await new Promise((resolve) => setTimeout(resolve, 10));
// Change concurrency - should clear pending tasks via clearAllPendingTasks
queue.setConcurrency(3);
// Handle all promises
const results = await Promise.allSettled([
job1Promise,
job2Promise,
job3Promise,
]);
// job1 should succeed (it was processing)
const job1Result = results[0];
expect(job1Result.status).toBe("fulfilled");
// Pending jobs should be rejected (job2 and job3 were in queue when cleared)
const job2Result = results[1];
const job3Result = results[2];
// At least one of the pending jobs should be rejected
const rejectedCount = [job2Result, job3Result].filter(
(r) => r && r.status === "rejected",
).length;
expect(rejectedCount).toBeGreaterThan(0);
// Verify rejection messages
if (job2Result && job2Result.status === "rejected") {
expect(job2Result.reason.message).toBe(
"Concurrency changed - queue cleared",
);
}
if (job3Result && job3Result.status === "rejected") {
expect(job3Result.reason.message).toBe(
"Concurrency changed - queue cleared",
);
}
await new Promise((resolve) => setTimeout(resolve, 100));
// job1 should have processed, others may or may not depending on timing
expect(processed.length).toBeGreaterThanOrEqual(1);
expect(processed).toContain("job1");
});
it("should allow new jobs after concurrency change", async () => {
const queue = new GroupedQueue<{ id: string }>(1);
const processed: string[] = [];
queue.setHandler(async (data) => {
await new Promise((resolve) => setTimeout(resolve, 50));
processed.push(data.id);
});
// Add job with concurrency 1
const job1Promise = queue.add("group1", { id: "job1" });
const job2Promise = queue.add("group1", { id: "job2" });
// Wait for job1 to start (it gets shifted from tasks)
await new Promise((resolve) => setTimeout(resolve, 10));
// Change concurrency to 3 - this calls clearAllPendingTasks internally
queue.setConcurrency(3);
// Handle all promises
const results = await Promise.allSettled([job1Promise, job2Promise]);
// job1 should succeed (it was processing)
const job1Result = results[0];
expect(job1Result.status).toBe("fulfilled");
// job2 should be rejected (it was in queue when cleared)
const job2Result = results[1];
if (job2Result && job2Result.status === "rejected") {
expect(job2Result.reason.message).toBe(
"Concurrency changed - queue cleared",
);
} else {
// If job2 wasn't rejected, it means it started processing before clear
// This is acceptable as it's a timing issue
}
// Add new jobs after concurrency change - they should work
await Promise.all([
queue.add("group2", { id: "job3" }),
queue.add("group3", { id: "job4" }),
]);
await new Promise((resolve) => setTimeout(resolve, 100));
// job1, job3, and job4 should have processed
expect(processed.length).toBeGreaterThanOrEqual(2);
expect(processed).toContain("job1");
});
});
});

View File

@@ -0,0 +1,313 @@
import { beforeEach, describe, expect, it } from "vitest";
import { QueueManager } from "../../server/queues/queue-manager";
describe("QueueManager", () => {
let manager: QueueManager;
beforeEach(() => {
manager = new QueueManager();
});
describe("Queue creation and retrieval", () => {
it("should create a queue with default concurrency 1", () => {
const queue = manager.getQueue("test-queue");
expect(queue.getConcurrency()).toBe(1);
});
it("should create a queue with custom concurrency", () => {
const queue = manager.getQueue("test-queue", 5);
expect(queue.getConcurrency()).toBe(5);
});
it("should return the same queue instance for the same name", () => {
const queue1 = manager.getQueue("test-queue", 3);
const queue2 = manager.getQueue("test-queue", 5);
expect(queue1).toBe(queue2);
// Concurrency should remain as first set
expect(queue1.getConcurrency()).toBe(3);
});
it("should create different queues for different names", () => {
const queue1 = manager.getQueue("queue1", 2);
const queue2 = manager.getQueue("queue2", 4);
expect(queue1).not.toBe(queue2);
expect(queue1.getConcurrency()).toBe(2);
expect(queue2.getConcurrency()).toBe(4);
});
});
describe("Handler management", () => {
it("should set handler for a queue", async () => {
const processed: string[] = [];
manager.setHandler("test-queue", async (data: { id: string }) => {
processed.push(data.id);
});
await manager.add("test-queue", "group1", { id: "job1" });
await new Promise((resolve) => setTimeout(resolve, 50));
expect(processed).toEqual(["job1"]);
});
it("should handle different handlers for different queues", async () => {
const queue1Processed: string[] = [];
const queue2Processed: string[] = [];
manager.setHandler("queue1", async (data: { id: string }) => {
queue1Processed.push(data.id);
});
manager.setHandler("queue2", async (data: { id: string }) => {
queue2Processed.push(data.id);
});
await Promise.all([
manager.add("queue1", "group1", { id: "job1" }),
manager.add("queue2", "group1", { id: "job2" }),
]);
await new Promise((resolve) => setTimeout(resolve, 50));
expect(queue1Processed).toEqual(["job1"]);
expect(queue2Processed).toEqual(["job2"]);
});
});
describe("Job management", () => {
it("should add jobs to correct queue and group", async () => {
const processed: string[] = [];
manager.setHandler("test-queue", async (data: { id: string }) => {
processed.push(data.id);
});
await manager.add("test-queue", "group1", { id: "job1" });
await manager.add("test-queue", "group2", { id: "job2" });
await new Promise((resolve) => setTimeout(resolve, 50));
expect(processed).toContain("job1");
expect(processed).toContain("job2");
});
it("should create queue with concurrency when adding job", async () => {
const processed: string[] = [];
// Create queue with concurrency first (without handler)
manager.getQueue("new-queue", 3);
// Set handler
manager.setHandler("new-queue", async (data: { id: string }) => {
processed.push(data.id);
});
// Now add job - it should process
await manager.add("new-queue", "group1", { id: "job1" });
await new Promise((resolve) => setTimeout(resolve, 50));
const queue = manager.getQueue("new-queue");
expect(queue.getConcurrency()).toBe(3);
expect(processed).toEqual(["job1"]);
});
});
describe("Queue operations", () => {
it("should clear group in specific queue", async () => {
const processed: string[] = [];
manager.setHandler("test-queue", async (data: { id: string }) => {
await new Promise((resolve) => setTimeout(resolve, 100));
processed.push(data.id);
});
// Add jobs but don't await - they'll start processing
const job1Promise = manager.add("test-queue", "group1", { id: "job1" });
const job2Promise = manager.add("test-queue", "group1", { id: "job2" });
// Clear immediately - job1 might be processing, but job2 should be cleared
manager.clearGroup("test-queue", "group1");
// Use Promise.allSettled to handle both promises properly
const results = await Promise.allSettled([job1Promise, job2Promise]);
// job1 might succeed or fail depending on timing
// job2 should be rejected
const job2Result = results[1];
if (job2Result.status === "rejected") {
expect(job2Result.reason.message).toBe("Queue cleared");
}
await new Promise((resolve) => setTimeout(resolve, 150));
// Job1 might have processed, but job2 should not
expect(processed.length).toBeLessThanOrEqual(1);
});
it("should get group length for specific queue", async () => {
manager.setHandler("test-queue", async () => {
await new Promise((resolve) => setTimeout(resolve, 100));
});
// Add jobs without awaiting - check length immediately
const job1Promise = manager.add("test-queue", "group1", { id: "job1" });
const job2Promise = manager.add("test-queue", "group1", { id: "job2" });
// Check length immediately - at least one should be pending
// (job1 might be processing, but job2 should be pending)
const length = manager.getGroupLength("test-queue", "group1");
expect(length).toBeGreaterThanOrEqual(0);
// Wait for both to complete
await Promise.all([job1Promise, job2Promise]);
await new Promise((resolve) => setTimeout(resolve, 50));
expect(manager.getGroupLength("test-queue", "group1")).toBe(0);
});
it("should get total length for specific queue", async () => {
manager.setHandler("test-queue", async () => {
await new Promise((resolve) => setTimeout(resolve, 50));
});
// Add jobs without awaiting - check length immediately
const promises = [
manager.add("test-queue", "group1", { id: "job1" }),
manager.add("test-queue", "group2", { id: "job2" }),
manager.add("test-queue", "group3", { id: "job3" }),
];
// Check length immediately - at least some should be pending
const length = manager.getTotalLength("test-queue");
expect(length).toBeGreaterThanOrEqual(0);
// Wait for all to complete
await Promise.all(promises);
await new Promise((resolve) => setTimeout(resolve, 100));
expect(manager.getTotalLength("test-queue")).toBe(0);
});
it("should check if queue is idle", async () => {
manager.setHandler("test-queue", async () => {
await new Promise((resolve) => setTimeout(resolve, 50));
});
expect(manager.isIdle("test-queue")).toBe(true);
await manager.add("test-queue", "group1", { id: "job1" });
await new Promise((resolve) => setTimeout(resolve, 100));
expect(manager.isIdle("test-queue")).toBe(true);
});
});
describe("Queue lifecycle", () => {
it("should close a specific queue", async () => {
manager.setHandler("test-queue", async () => {
await new Promise((resolve) => setTimeout(resolve, 100));
});
// Add first job and wait a bit to ensure it starts processing
const job1Promise = manager.add("test-queue", "group1", { id: "job1" });
// Add second job without awaiting
const job2Promise = manager.add("test-queue", "group1", { id: "job2" });
// Wait a tiny bit to ensure job2 is queued
await new Promise((resolve) => setTimeout(resolve, 10));
// Close queue - job2 should be rejected
await manager.closeQueue("test-queue");
// Use Promise.allSettled to handle both promises properly
const results = await Promise.allSettled([job1Promise, job2Promise]);
// job1 might succeed or fail depending on timing
// job2 should be rejected
const job2Result = results[1];
if (job2Result.status === "rejected") {
expect(job2Result.reason.message).toBe("Queue closed");
}
expect(manager.getQueueNames()).not.toContain("test-queue");
});
it("should close all queues", async () => {
manager.setHandler("queue1", async () => {
await new Promise((resolve) => setTimeout(resolve, 100));
});
manager.setHandler("queue2", async () => {
await new Promise((resolve) => setTimeout(resolve, 100));
});
await manager.add("queue1", "group1", { id: "job1" });
await manager.add("queue2", "group1", { id: "job2" });
await manager.closeAll();
expect(manager.getQueueNames()).toHaveLength(0);
});
it("should get all queue names", () => {
manager.getQueue("queue1");
manager.getQueue("queue2");
manager.getQueue("queue3");
const names = manager.getQueueNames();
expect(names).toContain("queue1");
expect(names).toContain("queue2");
expect(names).toContain("queue3");
expect(names).toHaveLength(3);
});
});
describe("Multiple queues with different concurrency", () => {
it("should handle multiple queues with different concurrency settings", async () => {
const queue1Processed: string[] = [];
const queue2Processed: string[] = [];
// Create queues with specific concurrency FIRST, before setting handlers
const queue1 = manager.getQueue("queue1", 1);
const queue2 = manager.getQueue("queue2", 3);
// Verify concurrency is set correctly before proceeding
expect(queue1.getConcurrency()).toBe(1);
expect(queue2.getConcurrency()).toBe(3);
manager.setHandler("queue1", async (data: { id: string }) => {
await new Promise((resolve) => setTimeout(resolve, 50));
queue1Processed.push(data.id);
});
manager.setHandler("queue2", async (data: { id: string }) => {
await new Promise((resolve) => setTimeout(resolve, 50));
queue2Processed.push(data.id);
});
// Queue1 with concurrency 1 (sequential)
await Promise.all([
manager.add("queue1", "app1", { id: "job1" }),
manager.add("queue1", "app2", { id: "job2" }),
]);
// Queue2 with concurrency 3 (parallel)
await Promise.all([
manager.add("queue2", "app1", { id: "job1" }),
manager.add("queue2", "app2", { id: "job2" }),
manager.add("queue2", "app3", { id: "job3" }),
]);
await new Promise((resolve) => setTimeout(resolve, 200));
expect(queue1Processed).toHaveLength(2);
expect(queue2Processed).toHaveLength(3);
// Verify concurrency settings are still correct
expect(manager.getQueue("queue1").getConcurrency()).toBe(1);
expect(manager.getQueue("queue2").getConcurrency()).toBe(3);
});
});
});

View File

@@ -0,0 +1,250 @@
import { beforeEach, describe, expect, it } from "vitest";
import type { DeploymentJob } from "../../server/queues/queue-types";
import {
getConcurrency,
myQueue,
setConcurrency,
} from "../../server/queues/queueSetup";
describe("queueSetup", () => {
beforeEach(() => {
// Reset concurrency to default (1) before each test
setConcurrency(1);
// Clear all pending tasks
myQueue.clearAllPendingTasks();
});
describe("getConcurrency", () => {
it("should return default concurrency of 1", () => {
const concurrency = getConcurrency();
expect(concurrency).toBe(1);
});
it("should return current concurrency after setting it", () => {
setConcurrency(3);
expect(getConcurrency()).toBe(3);
setConcurrency(5);
expect(getConcurrency()).toBe(5);
});
});
describe("setConcurrency", () => {
it("should set concurrency successfully", () => {
const clearedCount = setConcurrency(3);
expect(getConcurrency()).toBe(3);
expect(clearedCount).toBe(0); // No pending tasks to clear
});
it("should throw error for concurrency less than 1", () => {
expect(() => setConcurrency(0)).toThrow("Concurrency must be at least 1");
expect(() => setConcurrency(-1)).toThrow(
"Concurrency must be at least 1",
);
});
it("should return 0 cleared builds when no pending tasks", () => {
const clearedCount = setConcurrency(2);
expect(clearedCount).toBe(0);
expect(getConcurrency()).toBe(2);
});
it("should clear pending builds when concurrency changes", async () => {
const processed: string[] = [];
// Set handler
myQueue.setHandler(async (job: DeploymentJob) => {
await new Promise((resolve) => setTimeout(resolve, 100));
if (job.applicationType === "application") {
processed.push(job.applicationId);
} else if (job.applicationType === "compose") {
processed.push(job.composeId);
} else if (job.applicationType === "application-preview") {
processed.push(job.previewDeploymentId);
}
});
// Add jobs to different groups
const job1: DeploymentJob = {
applicationId: "app1",
titleLog: "Test",
descriptionLog: "Test",
type: "deploy",
applicationType: "application",
server: false,
};
const job2: DeploymentJob = {
applicationId: "app2",
titleLog: "Test",
descriptionLog: "Test",
type: "deploy",
applicationType: "application",
server: false,
};
const job3: DeploymentJob = {
applicationId: "app3",
titleLog: "Test",
descriptionLog: "Test",
type: "deploy",
applicationType: "application",
server: false,
};
// Add jobs without awaiting
const promise1 = myQueue.add("application:app1", job1);
const promise2 = myQueue.add("application:app2", job2);
const promise3 = myQueue.add("application:app3", job3);
// Wait for first job to start processing
await new Promise((resolve) => setTimeout(resolve, 10));
// Change concurrency - should clear pending builds
const clearedCount = setConcurrency(3);
// Should have cleared 2 pending builds (app2 and app3)
expect(clearedCount).toBe(2);
expect(getConcurrency()).toBe(3);
// Handle all promises - use allSettled to handle both resolved and rejected
const results = await Promise.allSettled([promise1, promise2, promise3]);
// job1 should succeed (it was processing), others should be rejected
const job1Result = results[0];
if (job1Result.status === "fulfilled") {
// Job1 completed successfully
}
// Pending jobs should be rejected
const job2Result = results[1];
const job3Result = results[2];
if (job2Result && job2Result.status === "rejected") {
expect(job2Result.reason.message).toBe(
"Concurrency changed - queue cleared",
);
}
if (job3Result && job3Result.status === "rejected") {
expect(job3Result.reason.message).toBe(
"Concurrency changed - queue cleared",
);
}
await new Promise((resolve) => setTimeout(resolve, 150));
// Only first job should have processed
expect(processed.length).toBeLessThanOrEqual(1);
});
it("should not clear builds when concurrency doesn't change", async () => {
// Set to 2
setConcurrency(2);
expect(getConcurrency()).toBe(2);
// Set to 2 again - should not clear anything
const clearedCount = setConcurrency(2);
expect(clearedCount).toBe(0);
expect(getConcurrency()).toBe(2);
});
it("should allow new jobs after concurrency change", async () => {
const processed: string[] = [];
myQueue.setHandler(async (job: DeploymentJob) => {
await new Promise((resolve) => setTimeout(resolve, 50));
if (job.applicationType === "application") {
processed.push(job.applicationId);
}
});
// Add job with concurrency 1
const job1: DeploymentJob = {
applicationId: "app1",
titleLog: "Test",
descriptionLog: "Test",
type: "deploy",
applicationType: "application",
server: false,
};
const job2: DeploymentJob = {
applicationId: "app2",
titleLog: "Test",
descriptionLog: "Test",
type: "deploy",
applicationType: "application",
server: false,
};
const promise1 = myQueue.add("application:app1", job1);
const promise2 = myQueue.add("application:app2", job2);
// Wait for first job to start
await new Promise((resolve) => setTimeout(resolve, 10));
// Change concurrency to 3
const clearedCount = setConcurrency(3);
expect(clearedCount).toBe(1); // app2 should be cleared
// Handle all promises - use allSettled to handle both resolved and rejected
const results = await Promise.allSettled([promise1, promise2]);
// job1 should succeed (it was processing)
const job1Result = results[0];
if (job1Result.status === "fulfilled") {
// Job1 completed successfully
}
// app2 should be rejected
const job2Result = results[1];
if (job2Result.status === "rejected") {
expect(job2Result.reason.message).toBe(
"Concurrency changed - queue cleared",
);
}
// Add new jobs after concurrency change - they should work
const job3: DeploymentJob = {
applicationId: "app3",
titleLog: "Test",
descriptionLog: "Test",
type: "deploy",
applicationType: "application",
server: false,
};
const job4: DeploymentJob = {
applicationId: "app4",
titleLog: "Test",
descriptionLog: "Test",
type: "deploy",
applicationType: "application",
server: false,
};
await Promise.all([
myQueue.add("application:app3", job3),
myQueue.add("application:app4", job4),
]);
await new Promise((resolve) => setTimeout(resolve, 150));
// app1, app3, and app4 should have processed
expect(processed.length).toBeGreaterThanOrEqual(2);
expect(processed).toContain("app1");
});
it("should handle multiple concurrency changes correctly", () => {
// Start at 1
expect(getConcurrency()).toBe(1);
// Change to 3
setConcurrency(3);
expect(getConcurrency()).toBe(3);
// Change to 5
setConcurrency(5);
expect(getConcurrency()).toBe(5);
// Change back to 1
setConcurrency(1);
expect(getConcurrency()).toBe(1);
});
});
});

View File

@@ -15,6 +15,7 @@ const baseApp: ApplicationNested = {
giteaId: "",
cleanCache: false,
applicationStatus: "done",
endpointSpecSwarm: null,
appName: "",
autoDeploy: true,
enableSubmodules: false,

View File

@@ -122,6 +122,22 @@ const NetworkSwarmSchema = z.array(
const LabelsSwarmSchema = z.record(z.string());
const EndpointPortConfigSwarmSchema = z
.object({
Protocol: z.string().optional(),
TargetPort: z.number().optional(),
PublishedPort: z.number().optional(),
PublishMode: z.string().optional(),
})
.strict();
const EndpointSpecSwarmSchema = z
.object({
Mode: z.string().optional(),
Ports: z.array(EndpointPortConfigSwarmSchema).optional(),
})
.strict();
const createStringToJSONSchema = (schema: z.ZodTypeAny) => {
return z
.string()
@@ -178,6 +194,9 @@ const addSwarmSettings = z.object({
labelsSwarm: createStringToJSONSchema(LabelsSwarmSchema).nullable(),
networkSwarm: createStringToJSONSchema(NetworkSwarmSchema).nullable(),
stopGracePeriodSwarm: z.bigint().nullable(),
endpointSpecSwarm: createStringToJSONSchema(
EndpointSpecSwarmSchema,
).nullable(),
});
type AddSwarmSettings = z.infer<typeof addSwarmSettings>;
@@ -234,6 +253,7 @@ export const AddSwarmSettings = ({ id, type }: Props) => {
labelsSwarm: null,
networkSwarm: null,
stopGracePeriodSwarm: null,
endpointSpecSwarm: null,
},
resolver: zodResolver(addSwarmSettings),
});
@@ -275,6 +295,9 @@ export const AddSwarmSettings = ({ id, type }: Props) => {
? JSON.stringify(data.networkSwarm, null, 2)
: null,
stopGracePeriodSwarm: normalizedStopGracePeriod,
endpointSpecSwarm: data.endpointSpecSwarm
? JSON.stringify(data.endpointSpecSwarm, null, 2)
: null,
});
}
}, [form, form.reset, data]);
@@ -296,6 +319,7 @@ export const AddSwarmSettings = ({ id, type }: Props) => {
labelsSwarm: data.labelsSwarm,
networkSwarm: data.networkSwarm,
stopGracePeriodSwarm: data.stopGracePeriodSwarm ?? null,
endpointSpecSwarm: data.endpointSpecSwarm,
})
.then(async () => {
toast.success("Swarm settings updated");
@@ -846,6 +870,67 @@ export const AddSwarmSettings = ({ id, type }: Props) => {
</FormItem>
)}
/>
<FormField
control={form.control}
name="endpointSpecSwarm"
render={({ field }) => (
<FormItem className="relative ">
<FormLabel>Endpoint Spec</FormLabel>
<TooltipProvider delayDuration={0}>
<Tooltip>
<TooltipTrigger asChild>
<FormDescription className="break-all w-fit flex flex-row gap-1 items-center">
Check the interface
<HelpCircle className="size-4 text-muted-foreground" />
</FormDescription>
</TooltipTrigger>
<TooltipContent
className="w-full z-[999]"
align="start"
side="bottom"
>
<code>
<pre>
{`{
Mode?: string | undefined;
Ports?: Array<{
Protocol?: string | undefined;
TargetPort?: number | undefined;
PublishedPort?: number | undefined;
PublishMode?: string | undefined;
}> | undefined;
}`}
</pre>
</code>
</TooltipContent>
</Tooltip>
</TooltipProvider>
<FormControl>
<CodeEditor
language="json"
placeholder={`{
"Mode": "dnsrr",
"Ports": [
{
"Protocol": "tcp",
"TargetPort": 5432,
"PublishedPort": 5432,
"PublishMode": "host"
}
]
}`}
className="h-[17rem] font-mono"
{...field}
value={field?.value || ""}
/>
</FormControl>
<pre>
<FormMessage />
</pre>
</FormItem>
)}
/>
<DialogFooter className="flex w-full flex-row justify-end md:col-span-2 m-0 sticky bottom-0 right-0 bg-muted border">
<Button
isLoading={isLoading}

View File

@@ -59,7 +59,13 @@ const mySchema = z.discriminatedUnion("type", [
z
.object({
type: z.literal("volume"),
volumeName: z.string().min(1, "Volume name required"),
volumeName: z
.string()
.min(1, "Volume name required")
.regex(
/^[a-zA-Z0-9][a-zA-Z0-9_.-]*$/,
"Invalid volume name. Use letters, numbers, '._-' and start with a letter/number.",
),
})
.merge(mountSchema),
z

View File

@@ -41,7 +41,13 @@ const mySchema = z.discriminatedUnion("type", [
z
.object({
type: z.literal("volume"),
volumeName: z.string().min(1, "Volume name required"),
volumeName: z
.string()
.min(1, "Volume name required")
.regex(
/^[a-zA-Z0-9][a-zA-Z0-9_.-]*$/,
"Invalid volume name. Use letters, numbers, '._-' and start with a letter/number.",
),
})
.merge(mountSchema),
z

View File

@@ -1,4 +1,12 @@
import { Clock, Loader2, RefreshCcw, RocketIcon, Settings } from "lucide-react";
import {
ChevronDown,
ChevronUp,
Clock,
Loader2,
RefreshCcw,
RocketIcon,
Settings,
} from "lucide-react";
import React, { useEffect, useMemo, useState } from "react";
import { toast } from "sonner";
import { AlertBlock } from "@/components/shared/alert-block";
@@ -80,6 +88,23 @@ export const ShowDeployments = ({
} = api.compose.cancelDeployment.useMutation();
const [url, setUrl] = React.useState("");
const [expandedDescriptions, setExpandedDescriptions] = useState<Set<string>>(
new Set(),
);
const MAX_DESCRIPTION_LENGTH = 200;
const truncateDescription = (description: string): string => {
if (description.length <= MAX_DESCRIPTION_LENGTH) {
return description;
}
const truncated = description.slice(0, MAX_DESCRIPTION_LENGTH);
const lastSpace = truncated.lastIndexOf(" ");
if (lastSpace > MAX_DESCRIPTION_LENGTH - 20 && lastSpace > 0) {
return `${truncated.slice(0, lastSpace)}...`;
}
return `${truncated}...`;
};
// Check for stuck deployment (more than 9 minutes) - only for the most recent deployment
const stuckDeployment = useMemo(() => {
@@ -217,118 +242,164 @@ export const ShowDeployments = ({
</div>
) : (
<div className="flex flex-col gap-4">
{deployments?.map((deployment, index) => (
<div
key={deployment.deploymentId}
className="flex items-center justify-between rounded-lg border p-4 gap-2"
>
<div className="flex flex-col">
<span className="flex items-center gap-4 font-medium capitalize text-foreground">
{index + 1}. {deployment.status}
<StatusTooltip
status={deployment?.status}
className="size-2.5"
/>
</span>
<span className="text-sm text-muted-foreground">
{deployment.title}
</span>
{deployment.description && (
<span className="break-all text-sm text-muted-foreground">
{deployment.description}
{deployments?.map((deployment, index) => {
const titleText = deployment?.title?.trim() || "";
const needsTruncation = titleText.length > MAX_DESCRIPTION_LENGTH;
const isExpanded = expandedDescriptions.has(
deployment.deploymentId,
);
return (
<div
key={deployment.deploymentId}
className="flex items-center justify-between rounded-lg border p-4 gap-2"
>
<div className="flex flex-col">
<span className="flex items-center gap-4 font-medium capitalize text-foreground">
{index + 1}. {deployment.status}
<StatusTooltip
status={deployment?.status}
className="size-2.5"
/>
</span>
)}
</div>
<div className="flex flex-col items-end gap-2">
<div className="text-sm capitalize text-muted-foreground flex items-center gap-2">
<DateTooltip date={deployment.createdAt} />
{deployment.startedAt && deployment.finishedAt && (
<Badge
variant="outline"
className="text-[10px] gap-1 flex items-center"
>
<Clock className="size-3" />
{formatDuration(
Math.floor(
(new Date(deployment.finishedAt).getTime() -
new Date(deployment.startedAt).getTime()) /
1000,
),
)}
</Badge>
)}
</div>
<div className="flex flex-row items-center gap-2">
{deployment.pid && deployment.status === "running" && (
<DialogAction
title="Kill Process"
description="Are you sure you want to kill the process?"
type="default"
onClick={async () => {
await killProcess({
deploymentId: deployment.deploymentId,
})
.then(() => {
toast.success("Process killed successfully");
})
.catch(() => {
toast.error("Error killing process");
});
}}
>
<Button
variant="destructive"
size="sm"
isLoading={isKillingProcess}
<div className="flex flex-col gap-1">
<span className="break-words text-sm text-muted-foreground whitespace-pre-wrap">
{isExpanded || !needsTruncation
? titleText
: truncateDescription(titleText)}
</span>
{needsTruncation && (
<button
type="button"
onClick={() => {
const next = new Set(expandedDescriptions);
if (next.has(deployment.deploymentId)) {
next.delete(deployment.deploymentId);
} else {
next.add(deployment.deploymentId);
}
setExpandedDescriptions(next);
}}
className="flex items-center gap-1 text-xs text-muted-foreground hover:text-foreground transition-colors w-fit mt-1 cursor-pointer"
aria-label={
isExpanded
? "Collapse commit message"
: "Expand commit message"
}
>
Kill Process
</Button>
</DialogAction>
)}
<Button
onClick={() => {
setActiveLog(deployment);
}}
>
View
</Button>
{isExpanded ? (
<>
<ChevronUp className="size-3" />
Show less
</>
) : (
<>
<ChevronDown className="size-3" />
Show more
</>
)}
</button>
)}
{/* Hash (from description) - shown in compact form */}
{deployment.description?.trim() && (
<span className="text-xs text-muted-foreground font-mono">
{deployment.description}
</span>
)}
</div>
</div>
<div className="flex flex-col items-end gap-2 max-w-[300px] w-full justify-start">
<div className="text-sm capitalize text-muted-foreground flex items-center gap-2">
<DateTooltip date={deployment.createdAt} />
{deployment.startedAt && deployment.finishedAt && (
<Badge
variant="outline"
className="text-[10px] gap-1 flex items-center"
>
<Clock className="size-3" />
{formatDuration(
Math.floor(
(new Date(deployment.finishedAt).getTime() -
new Date(deployment.startedAt).getTime()) /
1000,
),
)}
</Badge>
)}
</div>
{deployment?.rollback &&
deployment.status === "done" &&
type === "application" && (
<div className="flex flex-row items-center gap-2">
{deployment.pid && deployment.status === "running" && (
<DialogAction
title="Rollback to this deployment"
description="Are you sure you want to rollback to this deployment?"
title="Kill Process"
description="Are you sure you want to kill the process?"
type="default"
onClick={async () => {
await rollback({
rollbackId: deployment.rollback.rollbackId,
await killProcess({
deploymentId: deployment.deploymentId,
})
.then(() => {
toast.success(
"Rollback initiated successfully",
);
toast.success("Process killed successfully");
})
.catch(() => {
toast.error("Error initiating rollback");
toast.error("Error killing process");
});
}}
>
<Button
variant="secondary"
variant="destructive"
size="sm"
isLoading={isRollingBack}
isLoading={isKillingProcess}
>
<RefreshCcw className="size-4 text-primary group-hover:text-red-500" />
Rollback
Kill Process
</Button>
</DialogAction>
)}
<Button
onClick={() => {
setActiveLog(deployment);
}}
>
View
</Button>
{deployment?.rollback &&
deployment.status === "done" &&
type === "application" && (
<DialogAction
title="Rollback to this deployment"
description="Are you sure you want to rollback to this deployment?"
type="default"
onClick={async () => {
await rollback({
rollbackId: deployment.rollback.rollbackId,
})
.then(() => {
toast.success(
"Rollback initiated successfully",
);
})
.catch(() => {
toast.error("Error initiating rollback");
});
}}
>
<Button
variant="secondary"
size="sm"
isLoading={isRollingBack}
>
<RefreshCcw className="size-4 text-primary group-hover:text-red-500" />
Rollback
</Button>
</DialogAction>
)}
</div>
</div>
</div>
</div>
))}
);
})}
</div>
)}
<ShowDeployment

View File

@@ -79,7 +79,7 @@ export const ShowGeneralApplication = ({ applicationId }: Props) => {
>
<Button
variant="default"
isLoading={data?.applicationStatus === "running"}
// isLoading={data?.applicationStatus === "running"}
className="flex items-center gap-1.5 group focus-visible:ring-2 focus-visible:ring-offset-2"
>
<Tooltip>

View File

@@ -47,7 +47,13 @@ const formSchema = z
.object({
name: z.string().min(1, "Name is required"),
cronExpression: z.string().min(1, "Cron expression is required"),
volumeName: z.string().min(1, "Volume name is required"),
volumeName: z
.string()
.min(1, "Volume name is required")
.regex(
/^[a-zA-Z0-9][a-zA-Z0-9_.-]*$/,
"Invalid volume name. Use letters, numbers, '._-' and start with a letter/number.",
),
prefix: z.string(),
keepLatestCount: z.coerce
.number()

View File

@@ -1,6 +1,6 @@
import { zodResolver } from "@hookform/resolvers/zod";
import copy from "copy-to-clipboard";
import { debounce } from "lodash";
import _ from "lodash";
import {
CheckIcon,
ChevronsUpDown,
@@ -236,7 +236,7 @@ export const RestoreBackup = ({
const currentDatabaseType = form.watch("databaseType");
const metadata = form.watch("metadata");
const debouncedSetSearch = debounce((value: string) => {
const debouncedSetSearch = _.debounce((value: string) => {
setDebouncedSearchTerm(value);
}, 350);

View File

@@ -1,5 +1,5 @@
import { FancyAnsi } from "fancy-ansi";
import { escapeRegExp } from "lodash";
import _ from "lodash";
import { Badge } from "@/components/ui/badge";
import {
Tooltip,
@@ -47,7 +47,7 @@ export function TerminalLine({ log, noTimestamp, searchTerm }: LogLineProps) {
}
const htmlContent = fancyAnsi.toHtml(text);
const searchRegex = new RegExp(`(${escapeRegExp(term)})`, "gi");
const searchRegex = new RegExp(`(${_.escapeRegExp(term)})`, "gi");
const modifiedContent = htmlContent.replace(
searchRegex,

View File

@@ -14,6 +14,7 @@ import { useEffect, useMemo, useState } from "react";
import { toast } from "sonner";
import { BreadcrumbSidebar } from "@/components/shared/breadcrumb-sidebar";
import { DateTooltip } from "@/components/shared/date-tooltip";
import { FocusShortcutInput } from "@/components/shared/focus-shortcut-input";
import { StatusTooltip } from "@/components/shared/status-tooltip";
import {
AlertDialog,
@@ -44,7 +45,6 @@ import {
DropdownMenuSeparator,
DropdownMenuTrigger,
} from "@/components/ui/dropdown-menu";
import { FocusShortcutInput } from "@/components/shared/focus-shortcut-input";
import {
Select,
SelectContent,
@@ -52,12 +52,14 @@ import {
SelectTrigger,
SelectValue,
} from "@/components/ui/select";
import { TimeBadge } from "@/components/ui/time-badge";
import { api } from "@/utils/api";
import { HandleProject } from "./handle-project";
import { ProjectEnvironment } from "./project-environment";
export const ShowProjects = () => {
const utils = api.useUtils();
const { data: isCloud } = api.settings.isCloud.useQuery();
const { data, isLoading } = api.project.all.useQuery();
const { data: auth } = api.user.get.useQuery();
const { mutateAsync } = api.project.remove.useMutation();
@@ -135,6 +137,11 @@ export const ShowProjects = () => {
<BreadcrumbSidebar
list={[{ name: "Projects", href: "/dashboard/projects" }]}
/>
{!isCloud && (
<div className="absolute top-5 right-5">
<TimeBadge />
</div>
)}
<div className="w-full">
<Card className="h-full bg-sidebar p-2.5 rounded-xl ">
<div className="rounded-xl bg-background shadow-md ">
@@ -148,7 +155,6 @@ export const ShowProjects = () => {
Create and manage your projects
</CardDescription>
</CardHeader>
{(auth?.role === "owner" || auth?.canCreateProjects) && (
<div className="">
<HandleProject />
@@ -298,7 +304,13 @@ export const ShowProjects = () => {
<Link
className="space-x-4 text-xs cursor-pointer justify-between"
target="_blank"
href={`${domain.https ? "https" : "http"}://${domain.host}${domain.path}`}
href={`${
domain.https
? "https"
: "http"
}://${domain.host}${
domain.path
}`}
>
<span className="truncate">
{domain.host}
@@ -340,7 +352,13 @@ export const ShowProjects = () => {
<Link
className="space-x-4 text-xs cursor-pointer justify-between"
target="_blank"
href={`${domain.https ? "https" : "http"}://${domain.host}${domain.path}`}
href={`${
domain.https
? "https"
: "http"
}://${domain.host}${
domain.path
}`}
>
<span className="truncate">
{domain.host}

View File

@@ -15,6 +15,7 @@ import { api } from "@/utils/api";
import { ShowModalLogs } from "../../web-server/show-modal-logs";
import { TerminalModal } from "../../web-server/terminal-modal";
import { GPUSupportModal } from "../gpu-support-modal";
import { ChangeConcurrencyModal } from "../change-concurrency-modal";
export const ShowDokployActions = () => {
const { t } = useTranslation("settings");
@@ -101,6 +102,14 @@ export const ShowDokployActions = () => {
>
Reload Redis
</DropdownMenuItem>
<ChangeConcurrencyModal>
<DropdownMenuItem
className="cursor-pointer"
onSelect={(e) => e.preventDefault()}
>
Change Concurrency
</DropdownMenuItem>
</ChangeConcurrencyModal>
</DropdownMenuGroup>
</DropdownMenuContent>
</DropdownMenu>

View File

@@ -7,9 +7,11 @@ import {
DialogTrigger,
} from "@/components/ui/dialog";
import { DropdownMenuItem } from "@/components/ui/dropdown-menu";
import { Button } from "@/components/ui/button";
import { ShowStorageActions } from "./show-storage-actions";
import { ShowTraefikActions } from "./show-traefik-actions";
import { ToggleDockerCleanup } from "./toggle-docker-cleanup";
import { ChangeConcurrencyModal } from "../change-concurrency-modal";
interface Props {
serverId: string;
@@ -37,6 +39,16 @@ export const ShowServerActions = ({ serverId }: Props) => {
<ShowTraefikActions serverId={serverId} />
<ShowStorageActions serverId={serverId} />
<ToggleDockerCleanup serverId={serverId} />
<div className="col-span-2">
<ChangeConcurrencyModal
serverId={serverId}
trigger={
<Button variant="outline" className="w-full">
Change Concurrency
</Button>
}
/>
</div>
</div>
</DialogContent>
</Dialog>

View File

@@ -0,0 +1,180 @@
"use client";
import { InfoIcon, Loader2 } from "lucide-react";
import { useState } from "react";
import { toast } from "sonner";
import { Alert, AlertDescription } from "@/components/ui/alert";
import { Button } from "@/components/ui/button";
import {
Dialog,
DialogContent,
DialogDescription,
DialogFooter,
DialogHeader,
DialogTitle,
DialogTrigger,
} from "@/components/ui/dialog";
import { Input } from "@/components/ui/input";
import { Label } from "@/components/ui/label";
import { api } from "@/utils/api";
interface Props {
serverId?: string;
trigger?: React.ReactNode;
}
export const ChangeConcurrencyModal = ({ serverId, trigger }: Props) => {
const [isOpen, setIsOpen] = useState(false);
const [concurrency, setConcurrency] = useState<number | "">("");
const { data, isLoading: isLoadingCurrent } =
api.settings.getDeploymentConcurrency.useQuery(
{ serverId },
{
enabled: isOpen,
onSuccess: (data) => {
if (concurrency === "") {
setConcurrency(data.concurrency);
}
},
},
);
const { mutateAsync, isLoading } =
api.settings.setDeploymentConcurrency.useMutation();
const handleSubmit = async (e: React.FormEvent) => {
e.preventDefault();
if (
typeof concurrency !== "number" ||
concurrency < 1 ||
concurrency > 20
) {
toast.error("Concurrency must be between 1 and 20");
return;
}
try {
const result = await mutateAsync({ concurrency, serverId });
if (result.clearedBuilds > 0) {
toast.warning(
`Concurrency updated. ${result.clearedBuilds} pending build${result.clearedBuilds > 1 ? "s were" : " was"} cancelled.`,
);
} else {
toast.success("Concurrency updated successfully");
}
setIsOpen(false);
} catch (error) {
toast.error("Failed to update concurrency");
}
};
const serverType = serverId ? "Remote Server" : "Dokploy Server";
return (
<Dialog open={isOpen} onOpenChange={setIsOpen}>
<DialogTrigger asChild>
{trigger || (
<Button variant="outline" size="sm">
Change Concurrency
</Button>
)}
</DialogTrigger>
<DialogContent className="sm:max-w-md">
<DialogHeader>
<DialogTitle>Deployment Concurrency - {serverType}</DialogTitle>
<DialogDescription>
Configure how many deployments can run simultaneously on this
server.
</DialogDescription>
</DialogHeader>
<form onSubmit={handleSubmit} className="space-y-4">
<div className="space-y-2">
<Label htmlFor="concurrency">Concurrency</Label>
<Input
id="concurrency"
type="number"
min={1}
max={20}
value={concurrency}
onChange={(e) => {
const value = e.target.value;
setConcurrency(value === "" ? "" : Number.parseInt(value, 10));
}}
placeholder="Enter concurrency (1-20)"
disabled={isLoading || isLoadingCurrent}
/>
{isLoadingCurrent && (
<div className="flex items-center gap-2 text-sm text-muted-foreground">
<Loader2 className="h-4 w-4 animate-spin" />
Loading current concurrency...
</div>
)}
{!isLoadingCurrent && data && (
<p className="text-sm text-muted-foreground">
Current: {data.concurrency}
</p>
)}
</div>
<div className="space-y-3">
<Alert>
<InfoIcon className="h-4 w-4" />
<AlertDescription className="text-sm">
<div className="space-y-1 mt-1">
<p>
<strong>Default:</strong> 1 deployment at a time
(sequential)
</p>
<p>
<strong>Higher values:</strong> More deployments in
parallel, but will use more RAM and CPU resources.
</p>
{serverId && (
<p className="text-muted-foreground text-xs mt-2">
This setting applies to deployments on this remote server.
</p>
)}
{!serverId && (
<p className="text-muted-foreground text-xs mt-2">
This setting applies to deployments on the Dokploy server.
</p>
)}
</div>
</AlertDescription>
</Alert>
<Alert variant="destructive">
<InfoIcon className="h-4 w-4" />
<AlertDescription className="text-sm font-medium">
<strong>Warning:</strong> Changing concurrency will cancel all
pending builds. Currently running builds will continue, but
queued builds will be cancelled.
</AlertDescription>
</Alert>
</div>
<DialogFooter>
<Button
type="button"
variant="outline"
onClick={() => setIsOpen(false)}
disabled={isLoading}
>
Cancel
</Button>
<Button type="submit" disabled={isLoading || isLoadingCurrent}>
{isLoading ? (
<>
<Loader2 className="mr-2 h-4 w-4 animate-spin" />
Updating...
</>
) : (
"Update Concurrency"
)}
</Button>
</DialogFooter>
</form>
</DialogContent>
</Dialog>
);
};

View File

@@ -1,4 +1,3 @@
import type { findEnvironmentById } from "@dokploy/server/index";
import { zodResolver } from "@hookform/resolvers/zod";
import { useEffect, useState } from "react";
import { useForm } from "react-hook-form";
@@ -27,12 +26,10 @@ import {
FormMessage,
} from "@/components/ui/form";
import { Switch } from "@/components/ui/switch";
import { api } from "@/utils/api";
import { api, type RouterOutputs } from "@/utils/api";
type Environment = Omit<
Awaited<ReturnType<typeof findEnvironmentById>>,
"project"
>;
type Project = RouterOutputs["project"]["all"][number];
type Environment = Project["environments"][number];
export type Services = {
appName: string;
@@ -53,17 +50,16 @@ export type Services = {
};
export const extractServices = (data: Environment | undefined) => {
const applications: Services[] =
data?.applications.map((item) => ({
appName: item.appName,
name: item.name,
type: "application",
id: item.applicationId,
createdAt: item.createdAt,
status: item.applicationStatus,
description: item.description,
serverId: item.serverId,
})) || [];
const applications: Services[] = (data?.applications?.map((item) => ({
appName: item.appName,
name: item.name,
type: "application",
id: item.applicationId,
createdAt: item.createdAt,
status: item.applicationStatus,
description: item.description,
serverId: item.serverId,
})) ?? []) as Services[];
const mariadb: Services[] =
data?.mariadb.map((item) => ({
@@ -125,17 +121,16 @@ export const extractServices = (data: Environment | undefined) => {
serverId: item.serverId,
})) || [];
const compose: Services[] =
data?.compose.map((item) => ({
appName: item.appName,
name: item.name,
type: "compose",
id: item.composeId,
createdAt: item.createdAt,
status: item.composeStatus,
description: item.description,
serverId: item.serverId,
})) || [];
const compose: Services[] = (data?.compose?.map((item) => ({
appName: item.appName,
name: item.name,
type: "compose",
id: item.composeId,
createdAt: item.createdAt,
status: item.composeStatus,
description: item.description,
serverId: item.serverId,
})) ?? []) as Services[];
applications.push(
...mysql,

View File

@@ -26,6 +26,7 @@ import {
PieChart,
Server,
ShieldCheck,
Star,
Trash2,
User,
Users,
@@ -82,6 +83,7 @@ import { AddOrganization } from "../dashboard/organization/handle-organization";
import { DialogAction } from "../shared/dialog-action";
import { Logo } from "../shared/logo";
import { Button } from "../ui/button";
import { TimeBadge } from "../ui/time-badge";
import { UpdateServerButton } from "./update-server";
import { UserNav } from "./user-nav";
@@ -497,7 +499,6 @@ function SidebarLogo() {
const { data: isCloud } = api.settings.isCloud.useQuery();
const { data: user } = api.user.get.useQuery();
const { data: session } = authClient.useSession();
const {
data: organizations,
refetch,
@@ -505,6 +506,8 @@ function SidebarLogo() {
} = api.organization.all.useQuery();
const { mutateAsync: deleteOrganization, isLoading: isRemoving } =
api.organization.delete.useMutation();
const { mutateAsync: setDefaultOrganization, isLoading: isSettingDefault } =
api.organization.setDefault.useMutation();
const { isMobile } = useSidebar();
const { data: activeOrganization } = authClient.useActiveOrganization();
const _utils = api.useUtils();
@@ -594,66 +597,127 @@ function SidebarLogo() {
<DropdownMenuLabel className="text-xs text-muted-foreground">
Organizations
</DropdownMenuLabel>
{organizations?.map((org) => (
<div className="flex flex-row justify-between" key={org.name}>
<DropdownMenuItem
onClick={async () => {
await authClient.organization.setActive({
organizationId: org.id,
});
window.location.reload();
}}
className="w-full gap-2 p-2"
{organizations?.map((org) => {
const isDefault = org.members?.[0]?.isDefault ?? false;
return (
<div
className="flex flex-row justify-between"
key={org.name}
>
<div className="flex flex-col gap-4">{org.name}</div>
<div className="flex size-6 items-center justify-center rounded-sm border">
<Logo
className={cn(
"transition-all",
state === "collapsed" ? "size-6" : "size-10",
)}
logoUrl={org.logo ?? undefined}
/>
</div>
</DropdownMenuItem>
{org.ownerId === session?.user?.id && (
<DropdownMenuItem
onClick={async () => {
await authClient.organization.setActive({
organizationId: org.id,
});
window.location.reload();
}}
className="w-full gap-2 p-2"
>
<div className="flex flex-col gap-1">
<div className="flex items-center gap-2">
{org.name}
</div>
</div>
<div className="flex size-6 items-center justify-center rounded-sm border">
<Logo
className={cn(
"transition-all",
state === "collapsed" ? "size-6" : "size-10",
)}
logoUrl={org.logo ?? undefined}
/>
</div>
</DropdownMenuItem>
<div className="flex items-center gap-2">
<AddOrganization organizationId={org.id} />
<DialogAction
title="Delete Organization"
description="Are you sure you want to delete this organization?"
type="destructive"
onClick={async () => {
await deleteOrganization({
<Button
variant="ghost"
size="icon"
className={cn(
"group",
isDefault
? "hover:bg-yellow-500/10"
: "hover:bg-blue-500/10",
)}
isLoading={isSettingDefault && !isDefault}
disabled={isDefault}
onClick={async (e) => {
if (isDefault) return;
e.stopPropagation();
await setDefaultOrganization({
organizationId: org.id,
})
.then(() => {
refetch();
toast.success(
"Organization deleted successfully",
);
toast.success("Default organization updated");
})
.catch((error) => {
toast.error(
error?.message ||
"Error deleting organization",
"Error setting default organization",
);
});
}}
title={
isDefault
? "Default organization"
: "Set as default"
}
>
<Button
variant="ghost"
size="icon"
className="group hover:bg-red-500/10"
isLoading={isRemoving}
>
<Trash2 className="size-4 text-primary group-hover:text-red-500" />
</Button>
</DialogAction>
{isDefault ? (
<Star
fill="#eab308"
stroke="#eab308"
className="size-4 text-yellow-500"
/>
) : (
<Star
fill="none"
stroke="currentColor"
className="size-4 text-gray-400 group-hover:text-blue-500 transition-colors"
/>
)}
</Button>
{org.ownerId === session?.user?.id && (
<>
<AddOrganization organizationId={org.id} />
<DialogAction
title="Delete Organization"
description="Are you sure you want to delete this organization?"
type="destructive"
onClick={async () => {
await deleteOrganization({
organizationId: org.id,
})
.then(() => {
refetch();
toast.success(
"Organization deleted successfully",
);
})
.catch((error) => {
toast.error(
error?.message ||
"Error deleting organization",
);
});
}}
>
<Button
variant="ghost"
size="icon"
className="group hover:bg-red-500/10"
isLoading={isRemoving}
>
<Trash2 className="size-4 text-primary group-hover:text-red-500" />
</Button>
</DialogAction>
</>
)}
</div>
)}
</div>
))}
</div>
);
})}
{(user?.role === "owner" || isCloud) && (
<>
<DropdownMenuSeparator />
@@ -1062,6 +1126,7 @@ export default function Page({ children }: Props) {
</BreadcrumbList>
</Breadcrumb>
</div>
{!isCloud && <TimeBadge />}
</div>
</header>
)}

View File

@@ -67,9 +67,10 @@ export const Dropzone = React.forwardRef<HTMLDivElement, DropzoneProps>(
ref={inputRef}
type="file"
className={cn("hidden", className)}
onChange={(e: ChangeEvent<HTMLInputElement>) =>
onChange(e.target.files)
}
onChange={(e: ChangeEvent<HTMLInputElement>) => {
onChange(e.target.files);
e.target.value = "";
}}
/>
</div>
</CardContent>

View File

@@ -0,0 +1,58 @@
"use client";
import { useEffect, useState } from "react";
import { api } from "@/utils/api";
export function TimeBadge() {
const { data: serverTime } = api.server.getServerTime.useQuery(undefined);
const [time, setTime] = useState<Date | null>(null);
useEffect(() => {
if (serverTime?.time) {
setTime(new Date(serverTime.time));
}
}, [serverTime]);
useEffect(() => {
const timer = setInterval(() => {
setTime((prevTime) => {
if (!prevTime) return null;
const newTime = new Date(prevTime.getTime() + 1000);
return newTime;
});
}, 1000);
return () => {
clearInterval(timer);
};
}, []);
if (!time || !serverTime?.timezone) {
return null;
}
const getUtcOffset = (timeZone: string) => {
const date = new Date();
const utcDate = new Date(date.toLocaleString("en-US", { timeZone: "UTC" }));
const tzDate = new Date(date.toLocaleString("en-US", { timeZone }));
const offset = (tzDate.getTime() - utcDate.getTime()) / (1000 * 60 * 60);
const sign = offset >= 0 ? "+" : "-";
const hours = Math.floor(Math.abs(offset));
const minutes = (Math.abs(offset) * 60) % 60;
return `UTC${sign}${hours.toString().padStart(2, "0")}:${minutes
.toString()
.padStart(2, "0")}`;
};
return (
<div className="inline-flex items-center gap-2 rounded-md border px-2 py-1 text-xs sm:text-sm whitespace-nowrap max-w-full overflow-hidden">
<span className="hidden sm:inline">Server Time:</span>
<span className="font-medium tabular-nums">
{time.toLocaleTimeString()}
</span>
<span className="hidden sm:inline text-muted-foreground">
({serverTime.timezone} | {getUtcOffset(serverTime.timezone)})
</span>
</div>
);
}

View File

@@ -0,0 +1 @@
ALTER TABLE "member" ADD COLUMN "is_default" boolean DEFAULT false NOT NULL;

View File

@@ -0,0 +1,39 @@
ALTER TABLE "user_temp" RENAME TO "user";--> statement-breakpoint
ALTER TABLE "user" DROP CONSTRAINT "user_temp_email_unique";--> statement-breakpoint
ALTER TABLE "account" DROP CONSTRAINT "account_user_id_user_temp_id_fk";
--> statement-breakpoint
ALTER TABLE "apikey" DROP CONSTRAINT "apikey_user_id_user_temp_id_fk";
--> statement-breakpoint
ALTER TABLE "invitation" DROP CONSTRAINT "invitation_inviter_id_user_temp_id_fk";
--> statement-breakpoint
ALTER TABLE "member" DROP CONSTRAINT "member_user_id_user_temp_id_fk";
--> statement-breakpoint
ALTER TABLE "organization" DROP CONSTRAINT "organization_owner_id_user_temp_id_fk";
--> statement-breakpoint
ALTER TABLE "two_factor" DROP CONSTRAINT "two_factor_user_id_user_temp_id_fk";
--> statement-breakpoint
ALTER TABLE "backup" DROP CONSTRAINT "backup_userId_user_temp_id_fk";
--> statement-breakpoint
ALTER TABLE "git_provider" DROP CONSTRAINT "git_provider_userId_user_temp_id_fk";
--> statement-breakpoint
ALTER TABLE "schedule" DROP CONSTRAINT "schedule_userId_user_temp_id_fk";
--> statement-breakpoint
ALTER TABLE "session_temp" DROP CONSTRAINT "session_temp_user_id_user_temp_id_fk";
--> statement-breakpoint
ALTER TABLE "application" ADD COLUMN "endpointSpecSwarm" json;--> statement-breakpoint
ALTER TABLE "mariadb" ADD COLUMN "endpointSpecSwarm" json;--> statement-breakpoint
ALTER TABLE "mongo" ADD COLUMN "endpointSpecSwarm" json;--> statement-breakpoint
ALTER TABLE "mysql" ADD COLUMN "endpointSpecSwarm" json;--> statement-breakpoint
ALTER TABLE "postgres" ADD COLUMN "endpointSpecSwarm" json;--> statement-breakpoint
ALTER TABLE "redis" ADD COLUMN "endpointSpecSwarm" json;--> statement-breakpoint
ALTER TABLE "account" ADD CONSTRAINT "account_user_id_user_id_fk" FOREIGN KEY ("user_id") REFERENCES "public"."user"("id") ON DELETE cascade ON UPDATE no action;--> statement-breakpoint
ALTER TABLE "apikey" ADD CONSTRAINT "apikey_user_id_user_id_fk" FOREIGN KEY ("user_id") REFERENCES "public"."user"("id") ON DELETE cascade ON UPDATE no action;--> statement-breakpoint
ALTER TABLE "invitation" ADD CONSTRAINT "invitation_inviter_id_user_id_fk" FOREIGN KEY ("inviter_id") REFERENCES "public"."user"("id") ON DELETE cascade ON UPDATE no action;--> statement-breakpoint
ALTER TABLE "member" ADD CONSTRAINT "member_user_id_user_id_fk" FOREIGN KEY ("user_id") REFERENCES "public"."user"("id") ON DELETE cascade ON UPDATE no action;--> statement-breakpoint
ALTER TABLE "organization" ADD CONSTRAINT "organization_owner_id_user_id_fk" FOREIGN KEY ("owner_id") REFERENCES "public"."user"("id") ON DELETE cascade ON UPDATE no action;--> statement-breakpoint
ALTER TABLE "two_factor" ADD CONSTRAINT "two_factor_user_id_user_id_fk" FOREIGN KEY ("user_id") REFERENCES "public"."user"("id") ON DELETE cascade ON UPDATE no action;--> statement-breakpoint
ALTER TABLE "backup" ADD CONSTRAINT "backup_userId_user_id_fk" FOREIGN KEY ("userId") REFERENCES "public"."user"("id") ON DELETE no action ON UPDATE no action;--> statement-breakpoint
ALTER TABLE "git_provider" ADD CONSTRAINT "git_provider_userId_user_id_fk" FOREIGN KEY ("userId") REFERENCES "public"."user"("id") ON DELETE cascade ON UPDATE no action;--> statement-breakpoint
ALTER TABLE "schedule" ADD CONSTRAINT "schedule_userId_user_id_fk" FOREIGN KEY ("userId") REFERENCES "public"."user"("id") ON DELETE cascade ON UPDATE no action;--> statement-breakpoint
ALTER TABLE "session_temp" ADD CONSTRAINT "session_temp_user_id_user_id_fk" FOREIGN KEY ("user_id") REFERENCES "public"."user"("id") ON DELETE cascade ON UPDATE no action;--> statement-breakpoint
ALTER TABLE "user" ADD CONSTRAINT "user_email_unique" UNIQUE("email");

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

View File

@@ -834,6 +834,20 @@
"when": 1761415824484,
"tag": "0118_loose_anita_blake",
"breakpoints": true
},
{
"idx": 119,
"version": "7",
"when": 1762142756443,
"tag": "0119_bouncy_morbius",
"breakpoints": true
},
{
"idx": 120,
"version": "7",
"when": 1762632540024,
"tag": "0120_lame_captain_midlands",
"breakpoints": true
}
]
}

View File

@@ -120,7 +120,7 @@
"next": "^15.3.2",
"next-i18next": "^15.4.2",
"next-themes": "^0.2.1",
"node-os-utils": "1.3.7",
"node-os-utils": "2.0.1",
"node-pty": "1.0.0",
"node-schedule": "2.1.1",
"nodemailer": "6.9.14",
@@ -163,7 +163,6 @@
"@types/lodash": "4.17.4",
"@types/micromatch": "4.0.9",
"@types/node": "^18.19.104",
"@types/node-os-utils": "1.3.4",
"@types/node-schedule": "2.1.6",
"@types/nodemailer": "^6.4.17",
"@types/qrcode": "^1.5.5",

View File

@@ -12,6 +12,17 @@ import type { DeploymentJob } from "@/server/queues/queue-types";
import { myQueue } from "@/server/queues/queueSetup";
import { deploy } from "@/server/utils/deploy";
/**
* Helper function to get package_version from registry_package events
*/
const getPackageVersion = (headers: any, body: any) => {
const event = headers["x-github-event"];
if (event === "registry_package") {
return body.registry_package?.package_version;
}
return null;
};
export default async function handler(
req: NextApiRequest,
res: NextApiResponse,
@@ -46,21 +57,60 @@ export default async function handler(
}
const deploymentTitle = extractCommitMessage(req.headers, req.body);
const deploymentHash = extractHash(req.headers, req.body);
const deploymentHash = extractHash(req.headers, req.body);
const sourceType = application.sourceType;
if (sourceType === "docker") {
const applicationImageName = extractImageName(application.dockerImage);
const applicationDockerTag = extractImageTag(application.dockerImage);
const webhookImageName = extractImageNameFromRequest(
req.headers,
req.body,
);
const webhookDockerTag = extractImageTagFromRequest(
req.headers,
req.body,
);
if (
applicationDockerTag &&
webhookDockerTag &&
webhookDockerTag !== applicationDockerTag
) {
if (!applicationImageName) {
res.status(301).json({
message: "Application Docker Image Name Not Found",
});
return;
}
if (!webhookImageName) {
res.status(301).json({
message: "Webhook Docker Image Name Not Found",
});
return;
}
// Validate image name matches
if (webhookImageName !== applicationImageName) {
res.status(301).json({
message: `Application Image Name (${applicationImageName}) doesn't match request event payload Image Name (${webhookImageName}).`,
});
return;
}
if (!applicationDockerTag) {
res.status(301).json({
message: "Application Docker Tag Not Found",
});
return;
}
if (!webhookDockerTag) {
res.status(301).json({
message: "Webhook Docker Tag Not Found",
});
return;
}
if (webhookDockerTag !== applicationDockerTag) {
res.status(301).json({
message: `Application Image Tag (${applicationDockerTag}) doesn't match request event payload Image Tag (${webhookDockerTag}).`,
});
@@ -191,7 +241,7 @@ export default async function handler(
const jobData: DeploymentJob = {
applicationId: application.applicationId as string,
titleLog: deploymentTitle,
descriptionLog: `Hash: ${deploymentHash}`,
...(deploymentHash && { descriptionLog: `Hash: ${deploymentHash}` }),
type: "deploy",
applicationType: "application",
server: !!application.serverId,
@@ -203,12 +253,8 @@ export default async function handler(
return true;
}
await myQueue.add(
"deployments",
{ ...jobData },
{
removeOnComplete: true,
removeOnFail: true,
},
`application:${jobData.applicationId}`,
jobData,
);
} catch (error) {
res.status(400).json({ message: "Error deploying Application", error });
@@ -222,6 +268,39 @@ export default async function handler(
}
}
/**
* Return the image name without the tag
* Example: "my-image" => "my-image"
* Example: "my-image:latest" => "my-image"
* Example: "my-image:1.0.0" => "my-image"
* Example: "myregistryhost:5000/fedora/httpd:version1.0" => "myregistryhost:5000/fedora/httpd"
* @link https://docs.docker.com/reference/cli/docker/image/tag/
*/
export function extractImageName(dockerImage: string | null): string | null {
if (!dockerImage || typeof dockerImage !== "string") {
return null;
}
// Handle case where there's no tag (no colon or colon is part of port number)
const lastColonIndex = dockerImage.lastIndexOf(":");
if (lastColonIndex === -1) {
return dockerImage;
}
// Check if the part after the last colon looks like a tag (not a port number)
// Port numbers are typically 1-5 digits, tags are usually longer or contain letters
const afterColon = dockerImage.substring(lastColonIndex + 1);
const isPortNumber = /^\d{1,5}$/.test(afterColon);
// If it's a port number (like registry:5000/image), don't split
if (isPortNumber) {
return dockerImage;
}
// Otherwise, split at the last colon to get image name
return dockerImage.substring(0, lastColonIndex);
}
/**
* Return the last part of the image name, which is the tag
* Example: "my-image" => null
@@ -230,7 +309,7 @@ export default async function handler(
* Example: "myregistryhost:5000/fedora/httpd:version1.0" => "version1.0"
* @link https://docs.docker.com/reference/cli/docker/image/tag/
*/
function extractImageTag(dockerImage: string | null) {
export function extractImageTag(dockerImage: string | null) {
if (!dockerImage || typeof dockerImage !== "string") {
return null;
}
@@ -240,12 +319,78 @@ function extractImageTag(dockerImage: string | null) {
}
/**
* Extract the image name (without tag) from webhook request
* @link https://docs.docker.com/docker-hub/webhooks/#example-webhook-payload
* @link https://docs.github.com/en/webhooks/webhook-events-and-payloads#registry_package
*/
export const extractImageNameFromRequest = (
headers: any,
body: any,
): string | null => {
// GitHub Packages: registry_package events (container registry)
const packageVersion = getPackageVersion(headers, body);
if (packageVersion?.package_url) {
const packageUrl = packageVersion.package_url;
// Remove tag if present (everything after the last colon)
if (packageUrl.includes(":")) {
const lastColonIndex = packageUrl.lastIndexOf(":");
// Check if it's a port number (like registry:5000/image)
const afterColon = packageUrl.substring(lastColonIndex + 1);
const isPortNumber = /^\d{1,5}$/.test(afterColon);
if (isPortNumber) {
return packageUrl;
}
return packageUrl.substring(0, lastColonIndex);
}
return packageUrl;
}
// Docker Hub
if (headers["user-agent"]?.includes("Go-http-client")) {
if (body.repository) {
const repoName = body.repository.repo_name;
return `${repoName}`;
}
}
return null;
};
/**
* @link https://docs.docker.com/docker-hub/webhooks/#example-webhook-payload
* @link https://docs.github.com/en/webhooks/webhook-events-and-payloads#registry_package
*/
export const extractImageTagFromRequest = (
headers: any,
body: any,
): string | null => {
// GitHub Packages: registry_package events (container registry)
const packageVersion = getPackageVersion(headers, body);
if (packageVersion) {
// Try to get tag from container_metadata first (most reliable)
// Only use it if it's not empty and not the same as the version (digest)
const tagName = packageVersion.container_metadata?.tag?.name?.trim() || "";
if (
tagName &&
tagName !== packageVersion.version &&
!tagName.startsWith("sha256:")
) {
return tagName;
}
// Fallback: extract tag from package_url (e.g., "ghcr.io/owner/repo:tag")
if (packageVersion.package_url) {
const packageUrl = packageVersion.package_url;
// Handle case where package_url ends with colon (no tag)
if (packageUrl.endsWith(":")) {
return null;
}
const tagMatch = packageUrl.match(/:([^:]+)$/);
if (tagMatch?.[1]?.trim()) {
return tagMatch[1].trim();
}
}
}
// Docker Hub
if (headers["user-agent"]?.includes("Go-http-client")) {
if (body.push_data && body.repository) {
return body.push_data.tag;
@@ -255,6 +400,18 @@ export const extractImageTagFromRequest = (
};
export const extractCommitMessage = (headers: any, body: any) => {
// GitHub Packages: registry_package events (container tags)
const githubEvent = headers["x-github-event"];
if (githubEvent === "registry_package") {
const packageVersion = getPackageVersion(headers, body);
if (packageVersion) {
if (packageVersion.package_url) {
return `Docker GHCR image pushed: ${packageVersion.package_url}`;
}
return "Docker GHCR image pushed";
}
// If package_version is missing, fall through to default behavior
}
// GitHub
if (headers["x-github-event"]) {
return body.head_commit ? body.head_commit.message : "NEW COMMIT";
@@ -283,7 +440,7 @@ export const extractCommitMessage = (headers: any, body: any) => {
if (headers["user-agent"]?.includes("Go-http-client")) {
if (body.push_data && body.repository) {
return `Docker image pushed: ${body.repository.repo_name}:${body.push_data.tag} by ${body.push_data.pusher}`;
return `DockerHub image pushed: ${body.repository.repo_name}:${body.push_data.tag} by ${body.push_data.pusher}`;
}
}

View File

@@ -183,12 +183,8 @@ export default async function handler(
return true;
}
await myQueue.add(
"deployments",
{ ...jobData },
{
removeOnComplete: true,
removeOnFail: true,
},
`compose:${jobData.composeId}`,
jobData,
);
} catch (error) {
res.status(400).json({ message: "Error deploying Compose", error });

View File

@@ -132,12 +132,8 @@ export default async function handler(
continue;
}
await myQueue.add(
"deployments",
{ ...jobData },
{
removeOnComplete: true,
removeOnFail: true,
},
`application:${jobData.applicationId}`,
jobData,
);
}
@@ -170,12 +166,8 @@ export default async function handler(
}
await myQueue.add(
"deployments",
{ ...jobData },
{
removeOnComplete: true,
removeOnFail: true,
},
`compose:${jobData.composeId}`,
jobData,
);
}
@@ -250,12 +242,8 @@ export default async function handler(
continue;
}
await myQueue.add(
"deployments",
{ ...jobData },
{
removeOnComplete: true,
removeOnFail: true,
},
`application:${jobData.applicationId}`,
jobData,
);
}
@@ -296,12 +284,8 @@ export default async function handler(
}
await myQueue.add(
"deployments",
{ ...jobData },
{
removeOnComplete: true,
removeOnFail: true,
},
`compose:${jobData.composeId}`,
jobData,
);
}
@@ -495,12 +479,8 @@ export default async function handler(
continue;
}
await myQueue.add(
"deployments",
{ ...jobData },
{
removeOnComplete: true,
removeOnFail: true,
},
`preview:${jobData.previewDeploymentId}`,
jobData,
);
}
return res.status(200).json({ message: "Apps Deployed" });

View File

@@ -4,7 +4,7 @@ import { asc, eq } from "drizzle-orm";
import type { NextApiRequest, NextApiResponse } from "next";
import Stripe from "stripe";
import { db } from "@/server/db";
import { organization, server, users_temp } from "@/server/db/schema";
import { organization, server, user } from "@/server/db/schema";
const endpointSecret = process.env.STRIPE_WEBHOOK_SECRET!;
@@ -64,13 +64,13 @@ export default async function handler(
session.subscription as string,
);
await db
.update(users_temp)
.update(user)
.set({
stripeCustomerId: session.customer as string,
stripeSubscriptionId: session.subscription as string,
serversQuantity: subscription?.items?.data?.[0]?.quantity ?? 0,
})
.where(eq(users_temp.id, adminId))
.where(eq(user.id, adminId))
.returning();
const admin = await findUserById(adminId);
@@ -85,14 +85,12 @@ export default async function handler(
const newSubscription = event.data.object as Stripe.Subscription;
await db
.update(users_temp)
.update(user)
.set({
stripeSubscriptionId: newSubscription.id,
stripeCustomerId: newSubscription.customer as string,
})
.where(
eq(users_temp.stripeCustomerId, newSubscription.customer as string),
)
.where(eq(user.stripeCustomerId, newSubscription.customer as string))
.returning();
break;
@@ -102,14 +100,12 @@ export default async function handler(
const newSubscription = event.data.object as Stripe.Subscription;
await db
.update(users_temp)
.update(user)
.set({
stripeSubscriptionId: null,
serversQuantity: 0,
})
.where(
eq(users_temp.stripeCustomerId, newSubscription.customer as string),
);
.where(eq(user.stripeCustomerId, newSubscription.customer as string));
const admin = await findUserByStripeCustomerId(
newSubscription.customer as string,
@@ -135,24 +131,20 @@ export default async function handler(
if (newSubscription.status === "active") {
await db
.update(users_temp)
.update(user)
.set({
serversQuantity: newSubscription?.items?.data?.[0]?.quantity ?? 0,
})
.where(
eq(users_temp.stripeCustomerId, newSubscription.customer as string),
);
.where(eq(user.stripeCustomerId, newSubscription.customer as string));
const newServersQuantity = admin.serversQuantity;
await updateServersBasedOnQuantity(admin.id, newServersQuantity);
} else {
await disableServers(admin.id);
await db
.update(users_temp)
.update(user)
.set({ serversQuantity: 0 })
.where(
eq(users_temp.stripeCustomerId, newSubscription.customer as string),
);
.where(eq(user.stripeCustomerId, newSubscription.customer as string));
}
break;
@@ -172,11 +164,11 @@ export default async function handler(
}
await db
.update(users_temp)
.update(user)
.set({
serversQuantity: suscription?.items?.data?.[0]?.quantity ?? 0,
})
.where(eq(users_temp.stripeCustomerId, suscription.customer as string));
.where(eq(user.stripeCustomerId, suscription.customer as string));
const admin = await findUserByStripeCustomerId(
suscription.customer as string,
@@ -205,13 +197,11 @@ export default async function handler(
return res.status(400).send("Webhook Error: Admin not found");
}
await db
.update(users_temp)
.update(user)
.set({
serversQuantity: 0,
})
.where(
eq(users_temp.stripeCustomerId, newInvoice.customer as string),
);
.where(eq(user.stripeCustomerId, newInvoice.customer as string));
await disableServers(admin.id);
}
@@ -229,13 +219,13 @@ export default async function handler(
await disableServers(admin.id);
await db
.update(users_temp)
.update(user)
.set({
stripeCustomerId: null,
stripeSubscriptionId: null,
serversQuantity: 0,
})
.where(eq(users_temp.stripeCustomerId, customer.id));
.where(eq(user.stripeCustomerId, customer.id));
break;
}
@@ -262,10 +252,10 @@ const disableServers = async (userId: string) => {
};
const findUserByStripeCustomerId = async (stripeCustomerId: string) => {
const user = db.query.users_temp.findFirst({
where: eq(users_temp.stripeCustomerId, stripeCustomerId),
const userResult = await db.query.user.findFirst({
where: eq(user.stripeCustomerId, stripeCustomerId),
});
return user;
return userResult;
};
const activateServer = async (serverId: string) => {

View File

@@ -1,4 +1,4 @@
import type { findProjectById } from "@dokploy/server";
import type { findEnvironmentById } from "@dokploy/server";
import { validateRequest } from "@dokploy/server/lib/auth";
import { createServerSideHelpers } from "@trpc/react-query/server";
import {
@@ -102,6 +102,7 @@ import { api } from "@/utils/api";
export type Services = {
appName: string;
serverId?: string | null;
serverName?: string | null;
name: string;
type:
| "mariadb"
@@ -115,10 +116,10 @@ export type Services = {
id: string;
createdAt: string;
status?: "idle" | "running" | "done" | "error";
lastDeployDate?: Date | null;
};
type Project = Awaited<ReturnType<typeof findProjectById>>;
type Environment = Project["environments"][0];
type Environment = Awaited<ReturnType<typeof findEnvironmentById>>;
export const extractServicesFromEnvironment = (
environment: Environment | undefined,
@@ -128,16 +129,35 @@ export const extractServicesFromEnvironment = (
const allServices: Services[] = [];
const applications: Services[] =
environment.applications?.map((item) => ({
appName: item.appName,
name: item.name,
type: "application",
id: item.applicationId,
createdAt: item.createdAt,
status: item.applicationStatus,
description: item.description,
serverId: item.serverId,
})) || [];
environment.applications?.map((item) => {
// Get the most recent deployment date
let lastDeployDate: Date | null = null;
const deployments = (item as any).deployments;
if (deployments && deployments.length > 0) {
for (const deployment of deployments) {
const deployDate = new Date(
deployment.finishedAt ||
deployment.startedAt ||
deployment.createdAt,
);
if (!lastDeployDate || deployDate > lastDeployDate) {
lastDeployDate = deployDate;
}
}
}
return {
appName: item.appName,
name: item.name,
type: "application",
id: item.applicationId,
createdAt: item.createdAt,
status: item.applicationStatus,
description: item.description,
serverId: item.serverId,
serverName: item?.server?.name || null,
lastDeployDate,
};
}) || [];
const mariadb: Services[] =
environment.mariadb?.map((item) => ({
@@ -149,6 +169,7 @@ export const extractServicesFromEnvironment = (
status: item.applicationStatus,
description: item.description,
serverId: item.serverId,
serverName: item?.server?.name || null,
})) || [];
const postgres: Services[] =
@@ -161,6 +182,7 @@ export const extractServicesFromEnvironment = (
status: item.applicationStatus,
description: item.description,
serverId: item.serverId,
serverName: item?.server?.name || null,
})) || [];
const mongo: Services[] =
@@ -173,6 +195,7 @@ export const extractServicesFromEnvironment = (
status: item.applicationStatus,
description: item.description,
serverId: item.serverId,
serverName: item?.server?.name || null,
})) || [];
const redis: Services[] =
@@ -185,6 +208,7 @@ export const extractServicesFromEnvironment = (
status: item.applicationStatus,
description: item.description,
serverId: item.serverId,
serverName: item?.server?.name || null,
})) || [];
const mysql: Services[] =
@@ -197,19 +221,39 @@ export const extractServicesFromEnvironment = (
status: item.applicationStatus,
description: item.description,
serverId: item.serverId,
serverName: item?.server?.name || null,
})) || [];
const compose: Services[] =
environment.compose?.map((item) => ({
appName: item.appName,
name: item.name,
type: "compose",
id: item.composeId,
createdAt: item.createdAt,
status: item.composeStatus,
description: item.description,
serverId: item.serverId,
})) || [];
environment.compose?.map((item) => {
// Get the most recent deployment date
let lastDeployDate: Date | null = null;
const deployments = (item as any).deployments;
if (deployments && deployments.length > 0) {
for (const deployment of deployments) {
const deployDate = new Date(
deployment.finishedAt ||
deployment.startedAt ||
deployment.createdAt,
);
if (!lastDeployDate || deployDate > lastDeployDate) {
lastDeployDate = deployDate;
}
}
}
return {
appName: item.appName,
name: item.name,
type: "compose",
id: item.composeId,
createdAt: item.createdAt,
status: item.composeStatus,
description: item.description,
serverId: item.serverId,
serverName: item?.server?.name || null,
lastDeployDate,
};
}) || [];
allServices.push(
...applications,
@@ -237,9 +281,9 @@ const EnvironmentPage = (
const { data: auth } = api.user.get.useQuery();
const [sortBy, setSortBy] = useState<string>(() => {
if (typeof window !== "undefined") {
return localStorage.getItem("servicesSort") || "createdAt-desc";
return localStorage.getItem("servicesSort") || "lastDeploy-desc";
}
return "createdAt-desc";
return "lastDeploy-desc";
});
useEffect(() => {
@@ -261,10 +305,45 @@ const EnvironmentPage = (
comparison =
new Date(a.createdAt).getTime() - new Date(b.createdAt).getTime();
break;
case "lastDeploy": {
const aLastDeploy = a.lastDeployDate;
const bLastDeploy = b.lastDeployDate;
if (direction === "desc") {
// For "desc" (newest first): services with deployments first, then those without
if (!aLastDeploy && !bLastDeploy) {
comparison = 0;
} else if (!aLastDeploy) {
comparison = 1; // a (no deploy) goes after b (has deploy)
} else if (!bLastDeploy) {
comparison = -1; // a (has deploy) goes before b (no deploy)
} else {
// Both have deployments: newest first (negative if a is newer)
comparison = bLastDeploy.getTime() - aLastDeploy.getTime();
}
} else {
// For "asc" (oldest first): services with deployments first, then those without
if (!aLastDeploy && !bLastDeploy) {
comparison = 0;
} else if (!aLastDeploy) {
comparison = 1; // a (no deploy) goes after b (has deploy)
} else if (!bLastDeploy) {
comparison = -1; // a (has deploy) goes before b (no deploy)
} else {
// Both have deployments: oldest first
comparison = aLastDeploy.getTime() - bLastDeploy.getTime();
}
}
break;
}
default:
comparison = 0;
}
return direction === "asc" ? comparison : -comparison;
// For other fields, apply direction normally
if (field !== "lastDeploy") {
return direction === "asc" ? comparison : -comparison;
}
return comparison;
});
};
@@ -320,6 +399,7 @@ const EnvironmentPage = (
const [isDropdownOpen, setIsDropdownOpen] = useState(false);
const [isBulkDeleteDialogOpen, setIsBulkDeleteDialogOpen] = useState(false);
const [deleteVolumes, setDeleteVolumes] = useState(false);
const [selectedServerId, setSelectedServerId] = useState<string>("all");
const handleSelectAll = () => {
if (selectedServices.length === filteredServices.length) {
@@ -709,6 +789,27 @@ const EnvironmentPage = (
setIsBulkActionLoading(false);
};
// Get unique servers from services
const availableServers = useMemo(() => {
if (!applications) return [];
const servers = new Map<string, { serverId: string; serverName: string }>();
applications.forEach((service) => {
if (service.serverId && service.serverName) {
servers.set(service.serverId, {
serverId: service.serverId,
serverName: service.serverName,
});
}
});
return Array.from(servers.values());
}, [applications]);
// Check if there are services without a server (Dokploy server)
const hasServicesWithoutServer = useMemo(() => {
if (!applications) return false;
return applications.some((service) => !service.serverId);
}, [applications]);
const filteredServices = useMemo(() => {
if (!applications) return [];
const filtered = applications.filter(
@@ -717,10 +818,14 @@ const EnvironmentPage = (
service.description
?.toLowerCase()
.includes(searchQuery.toLowerCase())) &&
(selectedTypes.length === 0 || selectedTypes.includes(service.type)),
(selectedTypes.length === 0 || selectedTypes.includes(service.type)) &&
(selectedServerId === "" ||
selectedServerId === "all" ||
(selectedServerId === "dokploy-server" && !service.serverId) ||
service.serverId === selectedServerId),
);
return sortServices(filtered);
}, [applications, searchQuery, selectedTypes, sortBy]);
}, [applications, searchQuery, selectedTypes, selectedServerId, sortBy]);
const selectedServicesWithRunningStatus = useMemo(() => {
return filteredServices.filter(
@@ -1217,6 +1322,9 @@ const EnvironmentPage = (
<SelectValue placeholder="Sort by..." />
</SelectTrigger>
<SelectContent>
<SelectItem value="lastDeploy-desc">
Recently deployed
</SelectItem>
<SelectItem value="createdAt-desc">
Newest first
</SelectItem>
@@ -1291,6 +1399,39 @@ const EnvironmentPage = (
</Command>
</PopoverContent>
</Popover>
{(availableServers.length > 0 ||
hasServicesWithoutServer) && (
<Select
value={selectedServerId || "all"}
onValueChange={setSelectedServerId}
>
<SelectTrigger className="lg:w-[200px]">
<SelectValue placeholder="Filter by server..." />
</SelectTrigger>
<SelectContent>
<SelectItem value="all">All servers</SelectItem>
{hasServicesWithoutServer && (
<SelectItem value="dokploy-server">
<div className="flex items-center gap-2">
<ServerIcon className="size-4" />
<span>Dokploy server</span>
</div>
</SelectItem>
)}
{availableServers.map((server) => (
<SelectItem
key={server.serverId}
value={server.serverId}
>
<div className="flex items-center gap-2">
<ServerIcon className="size-4" />
<span>{server.serverName}</span>
</div>
</SelectItem>
))}
</SelectContent>
</Select>
)}
</div>
</div>
@@ -1396,7 +1537,15 @@ const EnvironmentPage = (
</CardTitle>
</CardHeader>
<CardFooter className="mt-auto">
<div className="space-y-1 text-sm">
<div className="space-y-1 text-sm w-full">
{service.serverName && (
<div className="flex items-center gap-1.5 text-xs text-muted-foreground mb-1">
<ServerIcon className="size-3" />
<span className="truncate">
{service.serverName}
</span>
</div>
)}
<DateTooltip date={service.createdAt}>
Created
</DateTooltip>

View File

@@ -1,6 +1,6 @@
import { findAdmin } from "@dokploy/server";
import { db } from "@dokploy/server/db";
import { users_temp } from "@dokploy/server/db/schema";
import { user } from "@dokploy/server/db/schema";
import { eq } from "drizzle-orm";
(async () => {
@@ -8,11 +8,11 @@ import { eq } from "drizzle-orm";
const result = await findAdmin();
const update = await db
.update(users_temp)
.update(user)
.set({
twoFactorEnabled: false,
})
.where(eq(users_temp.id, result.userId));
.where(eq(user.id, result.userId));
if (update) {
console.log("2FA reset successful");

View File

@@ -58,7 +58,10 @@ import {
applications,
} from "@/server/db/schema";
import type { DeploymentJob } from "@/server/queues/queue-types";
import { cleanQueuesByApplication, myQueue } from "@/server/queues/queueSetup";
import {
addJobAsync,
cleanQueuesByApplication,
} from "@/server/queues/queueSetup";
import { cancelDeployment, deploy } from "@/server/utils/deploy";
import { uploadFileSchema } from "@/utils/schema";
@@ -335,14 +338,9 @@ export const applicationRouter = createTRPCRouter({
await deploy(jobData);
return true;
}
await myQueue.add(
"deployments",
{ ...jobData },
{
removeOnComplete: true,
removeOnFail: true,
},
);
// Fire and forget - UI doesn't wait for deployment to complete
addJobAsync(`application:${jobData.applicationId}`, jobData);
return { success: true, message: "Deployment queued" };
}),
saveEnvironment: protectedProcedure
.input(apiSaveEnvironmentVariables)
@@ -700,14 +698,8 @@ export const applicationRouter = createTRPCRouter({
return true;
}
await myQueue.add(
"deployments",
{ ...jobData },
{
removeOnComplete: true,
removeOnFail: true,
},
);
addJobAsync(`application:${jobData.applicationId}`, jobData);
return { success: true, message: "Deployment queued" };
}),
cleanQueues: protectedProcedure
@@ -798,14 +790,8 @@ export const applicationRouter = createTRPCRouter({
return true;
}
await myQueue.add(
"deployments",
{ ...jobData },
{
removeOnComplete: true,
removeOnFail: true,
},
);
// Fire and forget - UI doesn't wait for deployment to complete
addJobAsync(`application:${jobData.applicationId}`, jobData);
return true;
}),
updateTraefikConfig: protectedProcedure

View File

@@ -3,13 +3,14 @@ import {
addNewService,
checkServiceAccess,
cloneCompose,
cloneComposeRemote,
createCommand,
createCompose,
createComposeByTemplate,
createDomain,
createMount,
deleteMount,
execAsync,
execAsyncRemote,
findComposeById,
findDomainsByComposeId,
findEnvironmentById,
@@ -58,7 +59,7 @@ import {
compose as composeTable,
} from "@/server/db/schema";
import type { DeploymentJob } from "@/server/queues/queue-types";
import { cleanQueuesByCompose, myQueue } from "@/server/queues/queueSetup";
import { addJobAsync, cleanQueuesByCompose } from "@/server/queues/queueSetup";
import { cancelDeployment, deploy } from "@/server/utils/deploy";
import { generatePassword } from "@/templates/utils";
import { createTRPCRouter, protectedProcedure, publicProcedure } from "../trpc";
@@ -245,6 +246,7 @@ export const composeRouter = createTRPCRouter({
});
}
await cleanQueuesByCompose(input.composeId);
return { success: true, message: "Queues cleaned successfully" };
}),
loadServices: protectedProcedure
@@ -301,10 +303,12 @@ export const composeRouter = createTRPCRouter({
message: "You are not authorized to fetch this compose",
});
}
const command = await cloneCompose(compose);
if (compose.serverId) {
await cloneComposeRemote(compose);
await execAsyncRemote(compose.serverId, command);
} else {
await cloneCompose(compose);
await execAsync(command);
}
return compose.sourceType;
} catch (err) {
@@ -397,14 +401,8 @@ export const composeRouter = createTRPCRouter({
await deploy(jobData);
return true;
}
await myQueue.add(
"deployments",
{ ...jobData },
{
removeOnComplete: true,
removeOnFail: true,
},
);
addJobAsync(`compose:${jobData.composeId}`, jobData);
return { success: true, message: "Deployment queued" };
}),
redeploy: protectedProcedure
.input(apiRedeployCompose)
@@ -432,14 +430,8 @@ export const composeRouter = createTRPCRouter({
await deploy(jobData);
return true;
}
await myQueue.add(
"deployments",
{ ...jobData },
{
removeOnComplete: true,
removeOnFail: true,
},
);
addJobAsync(`compose:${jobData.composeId}`, jobData);
return { success: true, message: "Redeployment queued" };
}),
stop: protectedProcedure
.input(apiFindCompose)

View File

@@ -1,8 +1,8 @@
import {
createDiscordNotification,
createEmailNotification,
createLarkNotification,
createGotifyNotification,
createLarkNotification,
createNtfyNotification,
createSlackNotification,
createTelegramNotification,
@@ -11,16 +11,16 @@ import {
removeNotificationById,
sendDiscordNotification,
sendEmailNotification,
sendLarkNotification,
sendGotifyNotification,
sendLarkNotification,
sendNtfyNotification,
sendServerThresholdNotifications,
sendSlackNotification,
sendTelegramNotification,
updateDiscordNotification,
updateEmailNotification,
updateLarkNotification,
updateGotifyNotification,
updateLarkNotification,
updateNtfyNotification,
updateSlackNotification,
updateTelegramNotification,
@@ -38,29 +38,29 @@ import { db } from "@/server/db";
import {
apiCreateDiscord,
apiCreateEmail,
apiCreateLark,
apiCreateGotify,
apiCreateLark,
apiCreateNtfy,
apiCreateSlack,
apiCreateTelegram,
apiFindOneNotification,
apiTestDiscordConnection,
apiTestEmailConnection,
apiTestLarkConnection,
apiTestGotifyConnection,
apiTestLarkConnection,
apiTestNtfyConnection,
apiTestSlackConnection,
apiTestTelegramConnection,
apiUpdateDiscord,
apiUpdateEmail,
apiUpdateLark,
apiUpdateGotify,
apiUpdateLark,
apiUpdateNtfy,
apiUpdateSlack,
apiUpdateTelegram,
notifications,
server,
users_temp,
user,
} from "@/server/db/schema";
export const notificationRouter = createTRPCRouter({
@@ -359,9 +359,9 @@ export const notificationRouter = createTRPCRouter({
if (input.ServerType === "Dokploy") {
const result = await db
.select()
.from(users_temp)
.from(user)
.where(
sql`${users_temp.metricsConfig}::jsonb -> 'server' ->> 'token' = ${input.Token}`,
sql`${user.metricsConfig}::jsonb -> 'server' ->> 'token' = ${input.Token}`,
);
if (!result?.[0]?.id) {

View File

@@ -41,6 +41,11 @@ export const organizationRouter = createTRPCRouter({
});
}
// Check if this is the user's first organization
const existingMemberships = await db.query.member.findMany({
where: eq(member.userId, ctx.user.id),
});
await db.insert(member).values({
organizationId: result.id,
role: "owner",
@@ -63,6 +68,11 @@ export const organizationRouter = createTRPCRouter({
),
),
),
with: {
members: {
where: eq(member.userId, ctx.user.id),
},
},
});
return memberResult;
}),
@@ -184,4 +194,45 @@ export const organizationRouter = createTRPCRouter({
.delete(invitation)
.where(eq(invitation.id, input.invitationId));
}),
setDefault: protectedProcedure
.input(
z.object({
organizationId: z.string().min(1),
}),
)
.mutation(async ({ ctx, input }) => {
// Verify user is a member of this organization
const userMember = await db.query.member.findFirst({
where: and(
eq(member.organizationId, input.organizationId),
eq(member.userId, ctx.user.id),
),
});
if (!userMember) {
throw new TRPCError({
code: "FORBIDDEN",
message: "You are not a member of this organization",
});
}
// First, unset all defaults for this user
await db
.update(member)
.set({ isDefault: false })
.where(eq(member.userId, ctx.user.id));
// Then set this organization as default
await db
.update(member)
.set({ isDefault: true })
.where(
and(
eq(member.organizationId, input.organizationId),
eq(member.userId, ctx.user.id),
),
);
return { success: true };
}),
});

View File

@@ -383,6 +383,15 @@ export const serverRouter = createTRPCRouter({
const ip = await getPublicIpWithFallback();
return ip;
}),
getServerTime: protectedProcedure.query(() => {
if (IS_CLOUD) {
return null;
}
return {
time: new Date(),
timezone: Intl.DateTimeFormat().resolvedOptions().timeZone,
};
}),
getServerMetrics: protectedProcedure
.input(
z.object({

View File

@@ -862,4 +862,49 @@ export const settingsRouter = createTRPCRouter({
const ips = process.env.DOKPLOY_CLOUD_IPS?.split(",");
return ips;
}),
getDeploymentConcurrency: adminProcedure
.input(
z.object({
serverId: z.string().optional(),
}),
)
.query(async ({ input }) => {
// For now, remote servers use the same queue as dokploy server
// In the future, we could implement per-server queues
const { getConcurrency } = await import("@/server/queues/queueSetup");
return {
concurrency: getConcurrency(),
serverId: input.serverId,
};
}),
setDeploymentConcurrency: adminProcedure
.input(
z.object({
concurrency: z.number().int().min(1).max(20),
serverId: z.string().optional(),
}),
)
.mutation(async ({ input }) => {
// For now, remote servers use the same queue as dokploy server
// In the future, we could implement per-server queues
const { setConcurrency, getConcurrency } = await import(
"@/server/queues/queueSetup"
);
const currentConcurrency = getConcurrency();
const clearedCount = setConcurrency(input.concurrency);
const serverType = input.serverId ? "remote server" : "Dokploy server";
let message = `${serverType} deployment concurrency updated from ${currentConcurrency} to ${input.concurrency}. Changes take effect immediately.`;
if (clearedCount > 0) {
message += ` ${clearedCount} pending build${clearedCount > 1 ? "s were" : " was"} cancelled due to concurrency change.`;
}
return {
success: true,
message,
concurrency: input.concurrency,
serverId: input.serverId,
clearedBuilds: clearedCount,
};
}),
});

View File

@@ -4,6 +4,7 @@ import {
findNotificationById,
findOrganizationById,
findUserById,
getDokployUrl,
getUserByToken,
IS_CLOUD,
removeUserById,
@@ -419,11 +420,10 @@ export const userRouter = createTRPCRouter({
});
}
const admin = await findAdmin();
const host =
process.env.NODE_ENV === "development"
? "http://localhost:3000"
: admin.user.host;
: await getDokployUrl();
const inviteLink = `${host}/invitation?token=${input.invitationId}`;
const organization = await findOrganizationById(

View File

@@ -2,121 +2,83 @@ import {
deployApplication,
deployCompose,
deployPreviewApplication,
deployRemoteApplication,
deployRemoteCompose,
deployRemotePreviewApplication,
rebuildApplication,
rebuildCompose,
rebuildRemoteApplication,
rebuildRemoteCompose,
updateApplicationStatus,
updateCompose,
updatePreviewDeployment,
} from "@dokploy/server";
import { type Job, Worker } from "bullmq";
import type { DeploymentJob } from "./queue-types";
import { redisConfig } from "./redis-connection";
import { myQueue } from "./queueSetup";
export const deploymentWorker = new Worker(
"deployments",
async (job: Job<DeploymentJob>) => {
try {
if (job.data.applicationType === "application") {
await updateApplicationStatus(job.data.applicationId, "running");
// Set the handler for processing deployment jobs
console.log("Setting deployment queue handler");
myQueue.setHandler(async (job: DeploymentJob) => {
const jobId =
job.applicationType === "application"
? job.applicationId
: job.applicationType === "compose"
? job.composeId
: job.previewDeploymentId;
console.log("Handler called with job:", job.applicationType, jobId);
try {
if (job.applicationType === "application") {
await updateApplicationStatus(job.applicationId, "running");
if (job.data.server) {
if (job.data.type === "redeploy") {
await rebuildRemoteApplication({
applicationId: job.data.applicationId,
titleLog: job.data.titleLog,
descriptionLog: job.data.descriptionLog,
});
} else if (job.data.type === "deploy") {
await deployRemoteApplication({
applicationId: job.data.applicationId,
titleLog: job.data.titleLog,
descriptionLog: job.data.descriptionLog,
});
}
} else {
if (job.data.type === "redeploy") {
await rebuildApplication({
applicationId: job.data.applicationId,
titleLog: job.data.titleLog,
descriptionLog: job.data.descriptionLog,
});
} else if (job.data.type === "deploy") {
await deployApplication({
applicationId: job.data.applicationId,
titleLog: job.data.titleLog,
descriptionLog: job.data.descriptionLog,
});
}
}
} else if (job.data.applicationType === "compose") {
await updateCompose(job.data.composeId, {
composeStatus: "running",
if (job.type === "redeploy") {
await rebuildApplication({
applicationId: job.applicationId,
titleLog: job.titleLog,
descriptionLog: job.descriptionLog,
});
if (job.data.server) {
if (job.data.type === "redeploy") {
await rebuildRemoteCompose({
composeId: job.data.composeId,
titleLog: job.data.titleLog,
descriptionLog: job.data.descriptionLog,
});
} else if (job.data.type === "deploy") {
await deployRemoteCompose({
composeId: job.data.composeId,
titleLog: job.data.titleLog,
descriptionLog: job.data.descriptionLog,
});
}
} else {
if (job.data.type === "deploy") {
await deployCompose({
composeId: job.data.composeId,
titleLog: job.data.titleLog,
descriptionLog: job.data.descriptionLog,
});
} else if (job.data.type === "redeploy") {
await rebuildCompose({
composeId: job.data.composeId,
titleLog: job.data.titleLog,
descriptionLog: job.data.descriptionLog,
});
}
}
} else if (job.data.applicationType === "application-preview") {
await updatePreviewDeployment(job.data.previewDeploymentId, {
previewStatus: "running",
} else if (job.type === "deploy") {
await deployApplication({
applicationId: job.applicationId,
titleLog: job.titleLog,
descriptionLog: job.descriptionLog,
});
}
} else if (job.applicationType === "compose") {
await updateCompose(job.composeId, {
composeStatus: "running",
});
if (job.type === "deploy") {
await deployCompose({
composeId: job.composeId,
titleLog: job.titleLog,
descriptionLog: job.descriptionLog,
});
} else if (job.type === "redeploy") {
await rebuildCompose({
composeId: job.composeId,
titleLog: job.titleLog,
descriptionLog: job.descriptionLog,
});
}
} else if (job.applicationType === "application-preview") {
await updatePreviewDeployment(job.previewDeploymentId, {
previewStatus: "running",
});
if (job.type === "deploy") {
await deployPreviewApplication({
applicationId: job.applicationId,
titleLog: job.titleLog,
descriptionLog: job.descriptionLog,
previewDeploymentId: job.previewDeploymentId,
});
if (job.data.server) {
if (job.data.type === "deploy") {
await deployRemotePreviewApplication({
applicationId: job.data.applicationId,
titleLog: job.data.titleLog,
descriptionLog: job.data.descriptionLog,
previewDeploymentId: job.data.previewDeploymentId,
});
}
} else {
if (job.data.type === "deploy") {
await deployPreviewApplication({
applicationId: job.data.applicationId,
titleLog: job.data.titleLog,
descriptionLog: job.data.descriptionLog,
previewDeploymentId: job.data.previewDeploymentId,
});
}
}
}
} catch (error) {
console.log("Error", error);
}
} catch (error) {
console.log("Error processing deployment job", error);
throw error; // Re-throw to let the queue handle retries if needed
}
});
// Export for compatibility (no longer needed but kept for imports)
export const deploymentWorker = {
run: () => {
// Queue starts processing automatically when jobs are added
console.log("Deployment queue handler initialized");
},
{
autorun: false,
connection: redisConfig,
},
);
};

View File

@@ -0,0 +1,256 @@
/**
* In-memory grouped queue implementation
* Each group processes one job at a time (FIFO per group)
* Multiple groups can process in parallel
*/
type Task<T> = {
data: T;
resolve: () => void;
reject: (error: Error) => void;
};
type GroupQueue<T> = {
tasks: Task<T>[];
processing: boolean;
};
export class GroupedQueue<T> {
private groups: Map<string, GroupQueue<T>> = new Map();
private handler?: (data: T) => Promise<void>;
private concurrency: number;
private activeGroups: Set<string> = new Set();
constructor(concurrency = 4) {
this.concurrency = concurrency;
}
/**
* Set the handler function that processes each job
*/
setHandler(handler: (data: T) => Promise<void>) {
this.handler = handler;
}
/**
* Add a job to a group queue
*/
async add(groupId: string, data: T): Promise<void> {
if (process.env.NODE_ENV !== "test") {
console.log(
`Adding job to group ${groupId}, handler set: ${!!this.handler}`,
);
}
return new Promise((resolve, reject) => {
if (!this.groups.has(groupId)) {
this.groups.set(groupId, {
tasks: [],
processing: false,
});
}
const group = this.groups.get(groupId)!;
group.tasks.push({
data,
resolve,
reject,
});
// Start processing if not already processing and under concurrency limit
if (!group.processing && this.activeGroups.size < this.concurrency) {
this.processGroup(groupId);
}
});
}
/**
* Process jobs in a group queue
*/
private async processGroup(groupId: string): Promise<void> {
const group = this.groups.get(groupId);
if (!group || group.processing) {
return;
}
// Wait for handler to be set if not available
if (!this.handler) {
if (process.env.NODE_ENV !== "test") {
console.log(`Handler not set yet for group ${groupId}, waiting...`);
}
// Retry after a short delay
setTimeout(() => {
if (this.handler && group.tasks.length > 0) {
this.processGroup(groupId);
}
}, 100);
return;
}
// Check concurrency limit
if (this.activeGroups.size >= this.concurrency) {
return;
}
group.processing = true;
this.activeGroups.add(groupId);
if (process.env.NODE_ENV !== "test") {
console.log(`Processing group ${groupId}, tasks: ${group.tasks.length}`);
}
while (group.tasks.length > 0) {
const task = group.tasks.shift()!;
try {
if (process.env.NODE_ENV !== "test") {
console.log(`Executing handler for group ${groupId}`);
}
await this.handler!(task.data);
task.resolve();
if (process.env.NODE_ENV !== "test") {
console.log(`Handler completed for group ${groupId}`);
}
} catch (error) {
if (process.env.NODE_ENV !== "test") {
console.error(`Handler error for group ${groupId}:`, error);
}
task.reject(error instanceof Error ? error : new Error(String(error)));
}
}
group.processing = false;
this.activeGroups.delete(groupId);
// Try to process another group if there are waiting groups
this.processNextGroup();
}
/**
* Process the next available group
*/
private processNextGroup(): void {
if (this.activeGroups.size >= this.concurrency) {
return;
}
// Find a group with pending tasks that's not currently processing
for (const [groupId, group] of this.groups.entries()) {
if (
!group.processing &&
group.tasks.length > 0 &&
!this.activeGroups.has(groupId)
) {
this.processGroup(groupId);
break;
}
}
}
/**
* Remove all tasks for a specific group
*/
clearGroup(groupId: string): void {
const group = this.groups.get(groupId);
if (group) {
// Reject all pending tasks
for (const task of group.tasks) {
task.reject(new Error("Queue cleared"));
}
group.tasks = [];
}
}
/**
* Clear all pending tasks across all groups
* This is useful when changing concurrency settings
* Note: This only clears tasks in the queue, not the currently executing task
*/
clearAllPendingTasks(): number {
let clearedCount = 0;
for (const [groupId, group] of this.groups.entries()) {
// Clear all pending tasks in the queue
// The currently executing task is not in group.tasks (it was already shifted)
if (group.tasks.length > 0) {
clearedCount += group.tasks.length;
for (const task of group.tasks) {
task.reject(new Error("Concurrency changed - queue cleared"));
}
group.tasks = [];
}
}
return clearedCount;
}
/**
* Get the number of pending tasks for a group
*/
getGroupLength(groupId: string): number {
return this.groups.get(groupId)?.tasks.length ?? 0;
}
/**
* Get total number of pending tasks across all groups
*/
getTotalLength(): number {
let total = 0;
for (const group of this.groups.values()) {
total += group.tasks.length;
}
return total;
}
/**
* Check if queue is idle (no active processing)
*/
isIdle(): boolean {
return this.activeGroups.size === 0;
}
/**
* Get the number of active groups (for testing)
*/
getActiveGroupsCount(): number {
return this.activeGroups.size;
}
/**
* Get the concurrency limit
*/
getConcurrency(): number {
return this.concurrency;
}
/**
* Set the concurrency limit dynamically
* This allows changing concurrency without recreating the queue
* WARNING: This will clear all pending tasks when concurrency changes
*/
setConcurrency(concurrency: number): void {
if (concurrency < 1) {
throw new Error("Concurrency must be at least 1");
}
const concurrencyChanged = this.concurrency !== concurrency;
this.concurrency = concurrency;
// If concurrency changed, clear all pending tasks
if (concurrencyChanged) {
this.clearAllPendingTasks();
}
// Process next group if we now have capacity
this.processNextGroup();
}
/**
* Close the queue and reject all pending tasks
*/
async close(): Promise<void> {
for (const [groupId, group] of this.groups.entries()) {
for (const task of group.tasks) {
task.reject(new Error("Queue closed"));
}
group.tasks = [];
}
this.groups.clear();
this.activeGroups.clear();
}
}

View File

@@ -0,0 +1,112 @@
/**
* Queue Manager - Manages multiple dynamic queues
* Each queue can have its own concurrency configuration
*/
import { GroupedQueue } from "./grouped-queue-wrapper";
export class QueueManager {
private queues: Map<string, GroupedQueue<any>> = new Map();
/**
* Get or create a queue with the specified name and concurrency
* Note: If queue already exists, concurrency parameter is ignored
*/
getQueue<T>(name: string, concurrency = 1): GroupedQueue<T> {
if (!this.queues.has(name)) {
this.queues.set(name, new GroupedQueue<T>(concurrency));
}
return this.queues.get(name) as GroupedQueue<T>;
}
/**
* Set handler for a specific queue
*/
setHandler<T>(queueName: string, handler: (data: T) => Promise<void>): void {
const queue = this.getQueue<T>(queueName);
queue.setHandler(handler);
}
/**
* Add a job to a specific queue and group
* If concurrency is provided and queue doesn't exist, creates it with that concurrency
*/
async add<T>(
queueName: string,
groupId: string,
data: T,
concurrency?: number,
): Promise<void> {
// If concurrency is provided and queue doesn't exist, create with that concurrency
if (concurrency !== undefined && !this.queues.has(queueName)) {
this.queues.set(queueName, new GroupedQueue<T>(concurrency));
}
const queue = this.getQueue<T>(queueName);
return queue.add(groupId, data);
}
/**
* Clear all tasks for a specific group in a queue
*/
clearGroup(queueName: string, groupId: string): void {
const queue = this.queues.get(queueName);
if (queue) {
queue.clearGroup(groupId);
}
}
/**
* Get the number of pending tasks for a group in a queue
*/
getGroupLength(queueName: string, groupId: string): number {
const queue = this.queues.get(queueName);
return queue ? queue.getGroupLength(groupId) : 0;
}
/**
* Get total number of pending tasks across all groups in a queue
*/
getTotalLength(queueName: string): number {
const queue = this.queues.get(queueName);
return queue ? queue.getTotalLength() : 0;
}
/**
* Check if a queue is idle
*/
isIdle(queueName: string): boolean {
const queue = this.queues.get(queueName);
return queue ? queue.isIdle() : true;
}
/**
* Close a specific queue
*/
async closeQueue(queueName: string): Promise<void> {
const queue = this.queues.get(queueName);
if (queue) {
await queue.close();
this.queues.delete(queueName);
}
}
/**
* Close all queues
*/
async closeAll(): Promise<void> {
const promises = Array.from(this.queues.keys()).map((name) =>
this.closeQueue(name),
);
await Promise.all(promises);
}
/**
* Get all queue names
*/
getQueueNames(): string[] {
return Array.from(this.queues.keys());
}
}
// Singleton instance
export const queueManager = new QueueManager();

View File

@@ -1,44 +1,110 @@
import { Queue } from "bullmq";
import { redisConfig } from "./redis-connection";
import { GroupedQueue } from "./grouped-queue-wrapper";
import type { DeploymentJob } from "./queue-types";
const myQueue = new Queue("deployments", {
connection: redisConfig,
});
// In-memory grouped queue: processes one job per group at a time
// Multiple groups can process in parallel (up to concurrency limit)
// Concurrency can be configured via DEPLOYMENT_QUEUE_CONCURRENCY env var (default: 1)
// or dynamically via setConcurrency() function
let DEPLOYMENT_CONCURRENCY = Number.parseInt(
process.env.DEPLOYMENT_QUEUE_CONCURRENCY || "1",
10,
);
process.on("SIGTERM", () => {
myQueue.close();
// Validate concurrency is at least 1
if (DEPLOYMENT_CONCURRENCY < 1) {
DEPLOYMENT_CONCURRENCY = 1;
}
const myQueue = new GroupedQueue<DeploymentJob>(DEPLOYMENT_CONCURRENCY);
// Initialize handler when this module is imported
// Use dynamic import to avoid circular dependency
// The handler will be set when deployments-queue.ts is imported
let handlerInitialized = false;
const initializeHandler = async () => {
if (!handlerInitialized) {
handlerInitialized = true;
// This will set the handler
await import("./deployments-queue");
}
};
// Initialize handler immediately (non-blocking)
void initializeHandler();
process.on("SIGTERM", async () => {
await myQueue.close();
process.exit(0);
});
myQueue.on("error", (error) => {
if ((error as any).code === "ECONNREFUSED") {
console.error(
"Make sure you have installed Redis and it is running.",
error,
);
}
});
export const cleanQueuesByApplication = async (applicationId: string) => {
const jobs = await myQueue.getJobs(["waiting", "delayed"]);
for (const job of jobs) {
if (job?.data?.applicationId === applicationId) {
await job.remove();
console.log(`Removed job ${job.id} for application ${applicationId}`);
}
}
const groupId = `application:${applicationId}`;
myQueue.clearGroup(groupId);
console.log(`Cleared queue for application ${applicationId}`);
};
export const cleanQueuesByCompose = async (composeId: string) => {
const jobs = await myQueue.getJobs(["waiting", "delayed"]);
const groupId = `compose:${composeId}`;
myQueue.clearGroup(groupId);
console.log(`Cleared queue for compose ${composeId}`);
};
for (const job of jobs) {
if (job?.data?.composeId === composeId) {
await job.remove();
console.log(`Removed job ${job.id} for compose ${composeId}`);
/**
* Add a job to the queue without awaiting (fire-and-forget)
* This allows the API to return immediately while the job processes in the background
* Errors are logged but don't block the response
*/
export const addJobAsync = (groupId: string, data: DeploymentJob): void => {
// Fire and forget - don't await, but handle errors
myQueue.add(groupId, data).catch((error) => {
console.error(`Failed to queue job for group ${groupId}:`, error);
});
};
/**
* Get the current deployment queue concurrency
*/
export const getConcurrency = (): number => {
return myQueue.getConcurrency();
};
/**
* Set the deployment queue concurrency dynamically
* This updates the queue's concurrency setting immediately
* WARNING: This will clear all pending builds when concurrency changes
* @returns The number of pending builds that were cleared
*/
export const setConcurrency = (concurrency: number): number => {
if (concurrency < 1) {
throw new Error("Concurrency must be at least 1");
}
const currentConcurrency = myQueue.getConcurrency();
const concurrencyChanged = currentConcurrency !== concurrency;
// Get count of pending tasks before clearing (setConcurrency will clear them)
let clearedCount = 0;
if (concurrencyChanged) {
// Get the count before setConcurrency clears them
clearedCount = myQueue.getTotalLength();
if (process.env.NODE_ENV !== "test") {
console.log(
`Concurrency changing from ${currentConcurrency} to ${concurrency}. Will clear ${clearedCount} pending builds.`,
);
}
}
// Update the stored concurrency value
DEPLOYMENT_CONCURRENCY = concurrency;
// Update the queue's concurrency dynamically (this will clear pending tasks)
myQueue.setConcurrency(concurrency);
if (process.env.NODE_ENV !== "test") {
console.log(`Deployment queue concurrency updated to ${concurrency}`);
}
return clearedCount;
};
export { myQueue };

View File

@@ -4,11 +4,11 @@ import {
createDefaultServerTraefikConfig,
createDefaultTraefikConfig,
IS_CLOUD,
initCancelDeployments,
initCronJobs,
initializeNetwork,
initSchedules,
initVolumeBackupsCronJobs,
initCancelDeployments,
sendDokployRestartNotifications,
setupDirectories,
} from "@dokploy/server";
@@ -66,6 +66,8 @@ void app.prepare().then(async () => {
console.log(`Server Started on: http://${HOST}:${PORT}`);
if (!IS_CLOUD) {
console.log("Starting Deployment Worker");
// Import the handler module to ensure it's initialized
await import("./queues/deployments-queue");
const { deploymentWorker } = await import("./queues/deployments-queue");
await deploymentWorker.run();
}

View File

@@ -2,6 +2,7 @@ import type http from "node:http";
import {
docker,
execAsync,
getHostSystemStats,
getLastAdvancedStatsFile,
recordAdvancedStats,
validateRequest,
@@ -49,6 +50,21 @@ export const setupDockerStatsMonitoringSocketServer = (
}
const intervalId = setInterval(async () => {
try {
// Special case: when monitoring "dokploy", get host system stats instead of container stats
if (appName === "dokploy") {
const stat = await getHostSystemStats();
await recordAdvancedStats(stat, appName);
const data = await getLastAdvancedStatsFile(appName);
ws.send(
JSON.stringify({
data,
}),
);
return;
}
const filter = {
status: ["running"],
...(appType === "application" && {

View File

@@ -22,7 +22,7 @@ import {
await initializeNetwork();
createDefaultTraefikConfig();
createDefaultServerTraefikConfig();
await execAsync("docker pull traefik:v3.5.0");
await execAsync("docker pull traefik:v3.6.1");
await initializeStandaloneTraefik();
await initializeRedis();
await initializePostgres();

View File

@@ -6,7 +6,7 @@
// boolean,
// } from "drizzle-orm/pg-core";
// export const users_temp = pgTable("users_temp", {
// export const user = pgTable("user", {
// id: text("id").primaryKey(),
// name: text("name").notNull(),
// email: text("email").notNull().unique(),
@@ -29,7 +29,7 @@
// userAgent: text("user_agent"),
// userId: text("user_id")
// .notNull()
// .references(() => users_temp.id, { onDelete: "cascade" }),
// .references(() => user.id, { onDelete: "cascade" }),
// activeOrganizationId: text("active_organization_id"),
// });
@@ -39,7 +39,7 @@
// providerId: text("provider_id").notNull(),
// userId: text("user_id")
// .notNull()
// .references(() => users_temp.id, { onDelete: "cascade" }),
// .references(() => user.id, { onDelete: "cascade" }),
// accessToken: text("access_token"),
// refreshToken: text("refresh_token"),
// idToken: text("id_token"),

View File

@@ -61,7 +61,7 @@
"lodash": "4.17.21",
"micromatch": "4.0.8",
"nanoid": "3.3.11",
"node-os-utils": "1.3.7",
"node-os-utils": "2.0.1",
"node-pty": "1.0.0",
"node-schedule": "2.1.1",
"nodemailer": "6.9.14",
@@ -88,7 +88,6 @@
"@types/lodash": "4.17.4",
"@types/micromatch": "4.0.9",
"@types/node": "^18.19.104",
"@types/node-os-utils": "1.3.4",
"@types/node-schedule": "2.1.6",
"@types/nodemailer": "^6.4.17",
"@types/qrcode": "^1.5.5",

View File

@@ -9,7 +9,7 @@ import {
import { nanoid } from "nanoid";
import { projects } from "./project";
import { server } from "./server";
import { users_temp } from "./user";
import { user } from "./user";
export const account = pgTable("account", {
id: text("id")
@@ -21,7 +21,7 @@ export const account = pgTable("account", {
providerId: text("provider_id").notNull(),
userId: text("user_id")
.notNull()
.references(() => users_temp.id, { onDelete: "cascade" }),
.references(() => user.id, { onDelete: "cascade" }),
accessToken: text("access_token"),
refreshToken: text("refresh_token"),
idToken: text("id_token"),
@@ -39,9 +39,9 @@ export const account = pgTable("account", {
});
export const accountRelations = relations(account, ({ one }) => ({
user: one(users_temp, {
user: one(user, {
fields: [account.userId],
references: [users_temp.id],
references: [user.id],
}),
}));
@@ -65,15 +65,15 @@ export const organization = pgTable("organization", {
metadata: text("metadata"),
ownerId: text("owner_id")
.notNull()
.references(() => users_temp.id, { onDelete: "cascade" }),
.references(() => user.id, { onDelete: "cascade" }),
});
export const organizationRelations = relations(
organization,
({ one, many }) => ({
owner: one(users_temp, {
owner: one(user, {
fields: [organization.ownerId],
references: [users_temp.id],
references: [user.id],
}),
servers: many(server),
projects: many(projects),
@@ -90,10 +90,11 @@ export const member = pgTable("member", {
.references(() => organization.id, { onDelete: "cascade" }),
userId: text("user_id")
.notNull()
.references(() => users_temp.id, { onDelete: "cascade" }),
.references(() => user.id, { onDelete: "cascade" }),
role: text("role").notNull().$type<"owner" | "member" | "admin">(),
createdAt: timestamp("created_at").notNull(),
teamId: text("team_id"),
isDefault: boolean("is_default").notNull().default(false),
// Permissions
canCreateProjects: boolean("canCreateProjects").notNull().default(false),
canAccessToSSHKeys: boolean("canAccessToSSHKeys").notNull().default(false),
@@ -133,9 +134,9 @@ export const memberRelations = relations(member, ({ one }) => ({
fields: [member.organizationId],
references: [organization.id],
}),
user: one(users_temp, {
user: one(user, {
fields: [member.userId],
references: [users_temp.id],
references: [user.id],
}),
}));
@@ -150,7 +151,7 @@ export const invitation = pgTable("invitation", {
expiresAt: timestamp("expires_at").notNull(),
inviterId: text("inviter_id")
.notNull()
.references(() => users_temp.id, { onDelete: "cascade" }),
.references(() => user.id, { onDelete: "cascade" }),
teamId: text("team_id"),
});
@@ -167,7 +168,7 @@ export const twoFactor = pgTable("two_factor", {
backupCodes: text("backup_codes").notNull(),
userId: text("user_id")
.notNull()
.references(() => users_temp.id, { onDelete: "cascade" }),
.references(() => user.id, { onDelete: "cascade" }),
});
export const apikey = pgTable("apikey", {
@@ -178,7 +179,7 @@ export const apikey = pgTable("apikey", {
key: text("key").notNull(),
userId: text("user_id")
.notNull()
.references(() => users_temp.id, { onDelete: "cascade" }),
.references(() => user.id, { onDelete: "cascade" }),
refillInterval: integer("refill_interval"),
refillAmount: integer("refill_amount"),
lastRefillAt: timestamp("last_refill_at"),
@@ -197,8 +198,8 @@ export const apikey = pgTable("apikey", {
});
export const apikeyRelations = relations(apikey, ({ one }) => ({
user: one(users_temp, {
user: one(user, {
fields: [apikey.userId],
references: [users_temp.id],
references: [user.id],
}),
}));

View File

@@ -28,6 +28,8 @@ import { server } from "./server";
import {
applicationStatus,
certificateType,
type EndpointSpecSwarm,
EndpointSpecSwarmSchema,
type HealthCheckSwarm,
HealthCheckSwarmSchema,
type LabelsSwarm,
@@ -167,6 +169,7 @@ export const applications = pgTable("application", {
labelsSwarm: json("labelsSwarm").$type<LabelsSwarm>(),
networkSwarm: json("networkSwarm").$type<NetworkSwarm[]>(),
stopGracePeriodSwarm: bigint("stopGracePeriodSwarm", { mode: "bigint" }),
endpointSpecSwarm: json("endpointSpecSwarm").$type<EndpointSpecSwarm>(),
//
replicas: integer("replicas").default(1).notNull(),
applicationStatus: applicationStatus("applicationStatus")
@@ -318,6 +321,7 @@ const createSchema = createInsertSchema(applications, {
previewLabels: z.array(z.string()).optional(),
cleanCache: z.boolean().optional(),
stopGracePeriodSwarm: z.bigint().nullable(),
endpointSpecSwarm: EndpointSpecSwarmSchema.nullable(),
});
export const apiCreateApplication = createSchema.pick({

View File

@@ -19,7 +19,7 @@ import { mariadb } from "./mariadb";
import { mongo } from "./mongo";
import { mysql } from "./mysql";
import { postgres } from "./postgres";
import { users_temp } from "./user";
import { user } from "./user";
export const databaseType = pgEnum("databaseType", [
"postgres",
"mariadb",
@@ -74,7 +74,7 @@ export const backups = pgTable("backup", {
mongoId: text("mongoId").references((): AnyPgColumn => mongo.mongoId, {
onDelete: "cascade",
}),
userId: text("userId").references(() => users_temp.id),
userId: text("userId").references(() => user.id),
// Only for compose backups
metadata: jsonb("metadata").$type<
| {
@@ -118,9 +118,9 @@ export const backupsRelations = relations(backups, ({ one, many }) => ({
fields: [backups.mongoId],
references: [mongo.mongoId],
}),
user: one(users_temp, {
user: one(user, {
fields: [backups.userId],
references: [users_temp.id],
references: [user.id],
}),
compose: one(compose, {
fields: [backups.composeId],

View File

@@ -8,7 +8,7 @@ import { bitbucket } from "./bitbucket";
import { gitea } from "./gitea";
import { github } from "./github";
import { gitlab } from "./gitlab";
import { users_temp } from "./user";
import { user } from "./user";
export const gitProviderType = pgEnum("gitProviderType", [
"github",
@@ -32,7 +32,7 @@ export const gitProvider = pgTable("git_provider", {
.references(() => organization.id, { onDelete: "cascade" }),
userId: text("userId")
.notNull()
.references(() => users_temp.id, { onDelete: "cascade" }),
.references(() => user.id, { onDelete: "cascade" }),
});
export const gitProviderRelations = relations(gitProvider, ({ one }) => ({
@@ -56,9 +56,9 @@ export const gitProviderRelations = relations(gitProvider, ({ one }) => ({
fields: [gitProvider.organizationId],
references: [organization.id],
}),
user: one(users_temp, {
user: one(user, {
fields: [gitProvider.userId],
references: [users_temp.id],
references: [user.id],
}),
}));

View File

@@ -9,6 +9,8 @@ import { mounts } from "./mount";
import { server } from "./server";
import {
applicationStatus,
type EndpointSpecSwarm,
EndpointSpecSwarmSchema,
type HealthCheckSwarm,
HealthCheckSwarmSchema,
type LabelsSwarm,
@@ -63,6 +65,7 @@ export const mariadb = pgTable("mariadb", {
labelsSwarm: json("labelsSwarm").$type<LabelsSwarm>(),
networkSwarm: json("networkSwarm").$type<NetworkSwarm[]>(),
stopGracePeriodSwarm: bigint("stopGracePeriodSwarm", { mode: "bigint" }),
endpointSpecSwarm: json("endpointSpecSwarm").$type<EndpointSpecSwarm>(),
replicas: integer("replicas").default(1).notNull(),
createdAt: text("createdAt")
.notNull()
@@ -130,6 +133,7 @@ const createSchema = createInsertSchema(mariadb, {
labelsSwarm: LabelsSwarmSchema.nullable(),
networkSwarm: NetworkSwarmSchema.nullable(),
stopGracePeriodSwarm: z.bigint().nullable(),
endpointSpecSwarm: EndpointSpecSwarmSchema.nullable(),
});
export const apiCreateMariaDB = createSchema

View File

@@ -16,6 +16,8 @@ import { mounts } from "./mount";
import { server } from "./server";
import {
applicationStatus,
type EndpointSpecSwarm,
EndpointSpecSwarmSchema,
type HealthCheckSwarm,
HealthCheckSwarmSchema,
type LabelsSwarm,
@@ -66,6 +68,7 @@ export const mongo = pgTable("mongo", {
labelsSwarm: json("labelsSwarm").$type<LabelsSwarm>(),
networkSwarm: json("networkSwarm").$type<NetworkSwarm[]>(),
stopGracePeriodSwarm: bigint("stopGracePeriodSwarm", { mode: "bigint" }),
endpointSpecSwarm: json("endpointSpecSwarm").$type<EndpointSpecSwarm>(),
replicas: integer("replicas").default(1).notNull(),
createdAt: text("createdAt")
.notNull()
@@ -127,6 +130,7 @@ const createSchema = createInsertSchema(mongo, {
labelsSwarm: LabelsSwarmSchema.nullable(),
networkSwarm: NetworkSwarmSchema.nullable(),
stopGracePeriodSwarm: z.bigint().nullable(),
endpointSpecSwarm: EndpointSpecSwarmSchema.nullable(),
});
export const apiCreateMongo = createSchema

View File

@@ -9,6 +9,8 @@ import { mounts } from "./mount";
import { server } from "./server";
import {
applicationStatus,
type EndpointSpecSwarm,
EndpointSpecSwarmSchema,
type HealthCheckSwarm,
HealthCheckSwarmSchema,
type LabelsSwarm,
@@ -61,6 +63,7 @@ export const mysql = pgTable("mysql", {
labelsSwarm: json("labelsSwarm").$type<LabelsSwarm>(),
networkSwarm: json("networkSwarm").$type<NetworkSwarm[]>(),
stopGracePeriodSwarm: bigint("stopGracePeriodSwarm", { mode: "bigint" }),
endpointSpecSwarm: json("endpointSpecSwarm").$type<EndpointSpecSwarm>(),
replicas: integer("replicas").default(1).notNull(),
createdAt: text("createdAt")
.notNull()
@@ -127,6 +130,7 @@ const createSchema = createInsertSchema(mysql, {
labelsSwarm: LabelsSwarmSchema.nullable(),
networkSwarm: NetworkSwarmSchema.nullable(),
stopGracePeriodSwarm: z.bigint().nullable(),
endpointSpecSwarm: EndpointSpecSwarmSchema.nullable(),
});
export const apiCreateMySql = createSchema

View File

@@ -9,6 +9,8 @@ import { mounts } from "./mount";
import { server } from "./server";
import {
applicationStatus,
type EndpointSpecSwarm,
EndpointSpecSwarmSchema,
type HealthCheckSwarm,
HealthCheckSwarmSchema,
type LabelsSwarm,
@@ -61,6 +63,7 @@ export const postgres = pgTable("postgres", {
labelsSwarm: json("labelsSwarm").$type<LabelsSwarm>(),
networkSwarm: json("networkSwarm").$type<NetworkSwarm[]>(),
stopGracePeriodSwarm: bigint("stopGracePeriodSwarm", { mode: "bigint" }),
endpointSpecSwarm: json("endpointSpecSwarm").$type<EndpointSpecSwarm>(),
replicas: integer("replicas").default(1).notNull(),
createdAt: text("createdAt")
.notNull()
@@ -120,6 +123,7 @@ const createSchema = createInsertSchema(postgres, {
labelsSwarm: LabelsSwarmSchema.nullable(),
networkSwarm: NetworkSwarmSchema.nullable(),
stopGracePeriodSwarm: z.bigint().nullable(),
endpointSpecSwarm: EndpointSpecSwarmSchema.nullable(),
});
export const apiCreatePostgres = createSchema

View File

@@ -5,10 +5,11 @@ import { nanoid } from "nanoid";
import { z } from "zod";
import { environments } from "./environment";
import { mounts } from "./mount";
import { projects } from "./project";
import { server } from "./server";
import {
applicationStatus,
type EndpointSpecSwarm,
EndpointSpecSwarmSchema,
type HealthCheckSwarm,
HealthCheckSwarmSchema,
type LabelsSwarm,
@@ -61,6 +62,7 @@ export const redis = pgTable("redis", {
labelsSwarm: json("labelsSwarm").$type<LabelsSwarm>(),
networkSwarm: json("networkSwarm").$type<NetworkSwarm[]>(),
stopGracePeriodSwarm: bigint("stopGracePeriodSwarm", { mode: "bigint" }),
endpointSpecSwarm: json("endpointSpecSwarm").$type<EndpointSpecSwarm>(),
replicas: integer("replicas").default(1).notNull(),
environmentId: text("environmentId")
@@ -110,6 +112,7 @@ const createSchema = createInsertSchema(redis, {
labelsSwarm: LabelsSwarmSchema.nullable(),
networkSwarm: NetworkSwarmSchema.nullable(),
stopGracePeriodSwarm: z.bigint().nullable(),
endpointSpecSwarm: EndpointSpecSwarmSchema.nullable(),
});
export const apiCreateRedis = createSchema

View File

@@ -7,7 +7,7 @@ import { applications } from "./application";
import { compose } from "./compose";
import { deployments } from "./deployment";
import { server } from "./server";
import { users_temp } from "./user";
import { user } from "./user";
import { generateAppName } from "./utils";
export const shellTypes = pgEnum("shellType", ["bash", "sh"]);
@@ -45,7 +45,7 @@ export const schedules = pgTable("schedule", {
serverId: text("serverId").references(() => server.serverId, {
onDelete: "cascade",
}),
userId: text("userId").references(() => users_temp.id, {
userId: text("userId").references(() => user.id, {
onDelete: "cascade",
}),
enabled: boolean("enabled").notNull().default(true),
@@ -69,9 +69,9 @@ export const schedulesRelations = relations(schedules, ({ one, many }) => ({
fields: [schedules.serverId],
references: [server.serverId],
}),
user: one(users_temp, {
user: one(user, {
fields: [schedules.userId],
references: [users_temp.id],
references: [user.id],
}),
deployments: many(deployments),
}));

View File

@@ -1,5 +1,5 @@
import { pgTable, text, timestamp } from "drizzle-orm/pg-core";
import { users_temp } from "./user";
import { user } from "./user";
// OLD TABLE
export const session = pgTable("session_temp", {
@@ -12,7 +12,7 @@ export const session = pgTable("session_temp", {
userAgent: text("user_agent"),
userId: text("user_id")
.notNull()
.references(() => users_temp.id, { onDelete: "cascade" }),
.references(() => user.id, { onDelete: "cascade" }),
impersonatedBy: text("impersonated_by"),
activeOrganizationId: text("active_organization_id"),
});

View File

@@ -74,6 +74,18 @@ export interface LabelsSwarm {
[name: string]: string;
}
export interface EndpointPortConfigSwarm {
Protocol?: string | undefined;
TargetPort?: number | undefined;
PublishedPort?: number | undefined;
PublishMode?: string | undefined;
}
export interface EndpointSpecSwarm {
Mode?: string | undefined;
Ports?: EndpointPortConfigSwarm[] | undefined;
}
export const HealthCheckSwarmSchema = z
.object({
Test: z.array(z.string()).optional(),
@@ -161,3 +173,19 @@ export const NetworkSwarmSchema = z.array(
);
export const LabelsSwarmSchema = z.record(z.string());
export const EndpointPortConfigSwarmSchema = z
.object({
Protocol: z.string().optional(),
TargetPort: z.number().optional(),
PublishedPort: z.number().optional(),
PublishMode: z.string().optional(),
})
.strict();
export const EndpointSpecSwarmSchema = z
.object({
Mode: z.string().optional(),
Ports: z.array(EndpointPortConfigSwarmSchema).optional(),
})
.strict();

View File

@@ -26,7 +26,7 @@ import { certificateType } from "./shared";
// OLD TABLE
// TEMP
export const users_temp = pgTable("user_temp", {
export const user = pgTable("user", {
id: text("id")
.notNull()
.primaryKey()
@@ -122,9 +122,9 @@ export const users_temp = pgTable("user_temp", {
serversQuantity: integer("serversQuantity").notNull().default(0),
});
export const usersRelations = relations(users_temp, ({ one, many }) => ({
export const usersRelations = relations(user, ({ one, many }) => ({
account: one(account, {
fields: [users_temp.id],
fields: [user.id],
references: [account.userId],
}),
organizations: many(organization),
@@ -134,7 +134,7 @@ export const usersRelations = relations(users_temp, ({ one, many }) => ({
schedules: many(schedules),
}));
const createSchema = createInsertSchema(users_temp, {
const createSchema = createInsertSchema(user, {
id: z.string().min(1),
isRegistered: z.boolean().optional(),
}).omit({

View File

@@ -165,6 +165,7 @@ const { handler, api } = betterAuth({
organizationId: organization?.id || "",
role: "owner",
createdAt: new Date(),
isDefault: true, // Mark first organization as default
});
});
}
@@ -174,9 +175,14 @@ const { handler, api } = betterAuth({
session: {
create: {
before: async (session) => {
// Find the default organization for this user
// Priority: 1) isDefault=true, 2) most recently created
const member = await db.query.member.findFirst({
where: eq(schema.member.userId, session.userId),
orderBy: desc(schema.member.createdAt),
orderBy: [
desc(schema.member.isDefault),
desc(schema.member.createdAt),
],
with: {
organization: true,
},
@@ -197,7 +203,7 @@ const { handler, api } = betterAuth({
updateAge: 60 * 60 * 24,
},
user: {
modelName: "users_temp",
modelName: "user",
additionalFields: {
role: {
type: "string",

View File

@@ -1,5 +1,5 @@
import { promises } from "node:fs";
import osUtils from "node-os-utils";
import { OSUtils } from "node-os-utils";
import { paths } from "../constants";
export interface Container {
@@ -38,22 +38,122 @@ export const recordAdvancedStats = async (
});
if (appName === "dokploy") {
const disk = await osUtils.drive.info("/");
const osutils = new OSUtils();
const diskResult = await osutils.disk.usageByMountPoint("/");
const diskUsage = disk.usedGb;
const diskTotal = disk.totalGb;
const diskUsedPercentage = disk.usedPercentage;
const diskFree = disk.freeGb;
if (diskResult.success && diskResult.data) {
const disk = diskResult.data;
const diskUsage = disk.used.toGB().toFixed(2);
const diskTotal = disk.total.toGB().toFixed(2);
const diskUsedPercentage = disk.usagePercentage;
const diskFree = disk.available.toGB().toFixed(2);
await updateStatsFile(appName, "disk", {
diskTotal: +diskTotal,
diskUsedPercentage: +diskUsedPercentage,
diskUsage: +diskUsage,
diskFree: +diskFree,
});
await updateStatsFile(appName, "disk", {
diskTotal: +diskTotal,
diskUsedPercentage: +diskUsedPercentage,
diskUsage: +diskUsage,
diskFree: +diskFree,
});
}
}
};
/**
* Get host system statistics using node-os-utils
* This is used when monitoring "dokploy" to show host stats instead of container stats
*/
export const getHostSystemStats = async (): Promise<Container> => {
const osutils = new OSUtils({
disk: {
includeStats: true, // Enable disk I/O statistics
},
});
// Get CPU usage
const cpuResult = await osutils.cpu.usage();
const cpuUsage = cpuResult.success ? cpuResult.data : 0;
// Get memory info
const memResult = await osutils.memory.info();
let memUsedGB = 0;
let memTotalGB = 0;
let memUsedPercent = 0;
if (memResult.success) {
memTotalGB = memResult.data.total.toGB();
memUsedGB = memResult.data.used.toGB();
memUsedPercent = memResult.data.usagePercentage;
}
// Get network stats from network.overview()
let netInputBytes = 0;
let netOutputBytes = 0;
const networkOverview = await osutils.network.overview();
if (networkOverview.success) {
netInputBytes = networkOverview.data.totalRxBytes.toBytes();
netOutputBytes = networkOverview.data.totalTxBytes.toBytes();
}
// Get Block I/O from disk.stats()
let blockReadBytes = 0;
let blockWriteBytes = 0;
const diskStats = await osutils.disk.stats();
if (diskStats.success && diskStats.data.length > 0) {
// Filter out virtual devices (loop, ram, sr, etc.) - only include real disk devices
const excludePatterns = [/^loop/, /^ram/, /^sr\d+$/, /^fd\d+$/];
for (const stat of diskStats.data) {
// Skip virtual devices
if (
stat.device &&
excludePatterns.some((pattern) => pattern.test(stat.device))
) {
continue;
}
// readBytes and writeBytes are DataSize objects with .toBytes() method
blockReadBytes += stat.readBytes.toBytes();
blockWriteBytes += stat.writeBytes.toBytes();
}
}
// Format values similar to docker stats
const formatBytes = (bytes: number): string => {
if (bytes >= 1024 * 1024 * 1024) {
return `${(bytes / (1024 * 1024 * 1024)).toFixed(2)}GiB`;
}
if (bytes >= 1024 * 1024) {
return `${(bytes / (1024 * 1024)).toFixed(2)}MiB`;
}
if (bytes >= 1024) {
return `${(bytes / 1024).toFixed(2)}KiB`;
}
return `${bytes}B`;
};
// Format memory usage similar to docker stats format: "used / total"
const memUsedFormatted = `${memUsedGB.toFixed(2)}GiB`;
const memTotalFormatted = `${memTotalGB.toFixed(2)}GiB`;
const memUsageFormatted = `${memUsedFormatted} / ${memTotalFormatted}`;
// Format network I/O
const netInputMb = netInputBytes / (1024 * 1024);
const netOutputMb = netOutputBytes / (1024 * 1024);
const netIOFormatted = `${netInputMb.toFixed(2)}MB / ${netOutputMb.toFixed(2)}MB`;
// Format Block I/O
const blockIOFormatted = `${formatBytes(blockReadBytes)} / ${formatBytes(blockWriteBytes)}`;
// Create a stat object compatible with recordAdvancedStats
return {
CPUPerc: `${cpuUsage.toFixed(2)}%`,
MemPerc: `${memUsedPercent.toFixed(2)}%`,
MemUsage: memUsageFormatted,
BlockIO: blockIOFormatted,
NetIO: netIOFormatted,
Container: "dokploy",
ID: "host-system",
Name: "dokploy",
};
};
export const getAdvancedStats = async (appName: string) => {
return {
cpu: await readStatsFile(appName, "cpu"),

View File

@@ -3,26 +3,26 @@ import {
invitation,
member,
organization,
users_temp,
user,
} from "@dokploy/server/db/schema";
import { TRPCError } from "@trpc/server";
import { eq } from "drizzle-orm";
import { IS_CLOUD } from "../constants";
export const findUserById = async (userId: string) => {
const user = await db.query.users_temp.findFirst({
where: eq(users_temp.id, userId),
const userResult = await db.query.user.findFirst({
where: eq(user.id, userId),
// with: {
// account: true,
// },
});
if (!user) {
if (!userResult) {
throw new TRPCError({
code: "NOT_FOUND",
message: "User not found",
});
}
return user;
return userResult;
};
export const findOrganizationById = async (organizationId: string) => {
@@ -64,7 +64,7 @@ export const findAdmin = async () => {
};
export const getUserByToken = async (token: string) => {
const user = await db.query.invitation.findFirst({
const userResult = await db.query.invitation.findFirst({
where: eq(invitation.id, token),
columns: {
id: true,
@@ -76,29 +76,29 @@ export const getUserByToken = async (token: string) => {
},
});
if (!user) {
if (!userResult) {
throw new TRPCError({
code: "NOT_FOUND",
message: "Invitation not found",
});
}
const userAlreadyExists = await db.query.users_temp.findFirst({
where: eq(users_temp.email, user?.email || ""),
const userAlreadyExists = await db.query.user.findFirst({
where: eq(user.email, userResult?.email || ""),
});
const { expiresAt, ...rest } = user;
const { expiresAt, ...rest } = userResult;
return {
...rest,
isExpired: user.expiresAt < new Date(),
isExpired: userResult.expiresAt < new Date(),
userAlreadyExists: !!userAlreadyExists,
};
};
export const removeUserById = async (userId: string) => {
await db
.delete(users_temp)
.where(eq(users_temp.id, userId))
.delete(user)
.where(eq(user.id, userId))
.returning()
.then((res) => res[0]);
};
@@ -110,7 +110,8 @@ export const getDokployUrl = async () => {
const admin = await findAdmin();
if (admin.user.host) {
return `https://${admin.user.host}`;
const protocol = admin.user.https ? "https" : "http";
return `${protocol}://${admin.user.host}`;
}
return `http://${admin.user.serverIp}:${process.env.PORT}`;
};

View File

@@ -7,45 +7,32 @@ import {
} from "@dokploy/server/db/schema";
import { getAdvancedStats } from "@dokploy/server/monitoring/utils";
import {
buildApplication,
getBuildCommand,
mechanizeDockerContainer,
} from "@dokploy/server/utils/builders";
import { sendBuildErrorNotifications } from "@dokploy/server/utils/notifications/build-error";
import { sendBuildSuccessNotifications } from "@dokploy/server/utils/notifications/build-success";
import { execAsyncRemote } from "@dokploy/server/utils/process/execAsync";
import {
cloneBitbucketRepository,
getBitbucketCloneCommand,
} from "@dokploy/server/utils/providers/bitbucket";
import {
buildDocker,
buildRemoteDocker,
} from "@dokploy/server/utils/providers/docker";
execAsync,
execAsyncRemote,
} from "@dokploy/server/utils/process/execAsync";
import { cloneBitbucketRepository } from "@dokploy/server/utils/providers/bitbucket";
import { buildRemoteDocker } from "@dokploy/server/utils/providers/docker";
import {
cloneGitRepository,
getCustomGitCloneCommand,
getGitCommitInfo,
} from "@dokploy/server/utils/providers/git";
import {
cloneGiteaRepository,
getGiteaCloneCommand,
} from "@dokploy/server/utils/providers/gitea";
import {
cloneGithubRepository,
getGithubCloneCommand,
} from "@dokploy/server/utils/providers/github";
import {
cloneGitlabRepository,
getGitlabCloneCommand,
} from "@dokploy/server/utils/providers/gitlab";
import { cloneGiteaRepository } from "@dokploy/server/utils/providers/gitea";
import { cloneGithubRepository } from "@dokploy/server/utils/providers/github";
import { cloneGitlabRepository } from "@dokploy/server/utils/providers/gitlab";
import { createTraefikConfig } from "@dokploy/server/utils/traefik/application";
import { TRPCError } from "@trpc/server";
import { eq } from "drizzle-orm";
import { encodeBase64 } from "../utils/docker/utils";
import { getDokployUrl } from "./admin";
import {
createDeployment,
createDeploymentPreview,
updateDeployment,
updateDeploymentStatus,
} from "./deployment";
import { type Domain, getDomainHost } from "./domain";
@@ -192,30 +179,31 @@ export const deployApplication = async ({
});
try {
let command = "set -e;";
if (application.sourceType === "github") {
await cloneGithubRepository({
...application,
logPath: deployment.logPath,
});
await buildApplication(application, deployment.logPath);
command += await cloneGithubRepository(application);
} else if (application.sourceType === "gitlab") {
await cloneGitlabRepository(application, deployment.logPath);
await buildApplication(application, deployment.logPath);
command += await cloneGitlabRepository(application);
} else if (application.sourceType === "gitea") {
await cloneGiteaRepository(application, deployment.logPath);
await buildApplication(application, deployment.logPath);
command += await cloneGiteaRepository(application);
} else if (application.sourceType === "bitbucket") {
await cloneBitbucketRepository(application, deployment.logPath);
await buildApplication(application, deployment.logPath);
} else if (application.sourceType === "docker") {
await buildDocker(application, deployment.logPath);
command += await cloneBitbucketRepository(application);
} else if (application.sourceType === "git") {
await cloneGitRepository(application, deployment.logPath);
await buildApplication(application, deployment.logPath);
} else if (application.sourceType === "drop") {
await buildApplication(application, deployment.logPath);
command += await cloneGitRepository(application);
} else if (application.sourceType === "docker") {
command += await buildRemoteDocker(application);
}
command += getBuildCommand(application);
const commandWithLog = `(${command}) >> ${deployment.logPath} 2>&1`;
if (application.serverId) {
await execAsyncRemote(application.serverId, commandWithLog);
} else {
await execAsync(commandWithLog);
}
await mechanizeDockerContainer(application);
await updateDeploymentStatus(deployment.deploymentId, "done");
await updateApplicationStatus(applicationId, "done");
@@ -239,6 +227,12 @@ export const deployApplication = async ({
domains: application.domains,
});
} catch (error) {
const command = `echo "Error occurred ❌, check the logs for details." >> ${deployment.logPath};`;
if (application.serverId) {
await execAsyncRemote(application.serverId, command);
} else {
await execAsync(command);
}
await updateDeploymentStatus(deployment.deploymentId, "error");
await updateApplicationStatus(applicationId, "error");
@@ -253,8 +247,19 @@ export const deployApplication = async ({
});
throw error;
}
} finally {
// Only extract commit info for non-docker sources
if (application.sourceType !== "docker") {
const commitInfo = await getGitCommitInfo(application);
if (commitInfo) {
await updateDeployment(deployment.deploymentId, {
title: commitInfo.message,
description: `Commit: ${commitInfo.hash}`,
});
}
}
}
return true;
};
@@ -276,129 +281,21 @@ export const rebuildApplication = async ({
});
try {
if (application.sourceType === "github") {
await buildApplication(application, deployment.logPath);
} else if (application.sourceType === "gitlab") {
await buildApplication(application, deployment.logPath);
} else if (application.sourceType === "bitbucket") {
await buildApplication(application, deployment.logPath);
} else if (application.sourceType === "docker") {
await buildDocker(application, deployment.logPath);
} else if (application.sourceType === "git") {
await buildApplication(application, deployment.logPath);
} else if (application.sourceType === "drop") {
await buildApplication(application, deployment.logPath);
}
await updateDeploymentStatus(deployment.deploymentId, "done");
await updateApplicationStatus(applicationId, "done");
} catch (error) {
await updateDeploymentStatus(deployment.deploymentId, "error");
await updateApplicationStatus(applicationId, "error");
throw error;
}
return true;
};
export const deployRemoteApplication = async ({
applicationId,
titleLog = "Manual deployment",
descriptionLog = "",
}: {
applicationId: string;
titleLog: string;
descriptionLog: string;
}) => {
const application = await findApplicationById(applicationId);
const buildLink = `${await getDokployUrl()}/dashboard/project/${application.environment.projectId}/environment/${application.environmentId}/services/application/${application.applicationId}?tab=deployments`;
const deployment = await createDeployment({
applicationId: applicationId,
title: titleLog,
description: descriptionLog,
});
try {
let command = "set -e;";
// Check case for docker only
command += getBuildCommand(application);
const commandWithLog = `(${command}) >> ${deployment.logPath} 2>&1`;
if (application.serverId) {
let command = "set -e;";
if (application.sourceType === "github") {
command += await getGithubCloneCommand({
...application,
serverId: application.serverId,
logPath: deployment.logPath,
});
} else if (application.sourceType === "gitlab") {
command += await getGitlabCloneCommand(application, deployment.logPath);
} else if (application.sourceType === "bitbucket") {
command += await getBitbucketCloneCommand(
application,
deployment.logPath,
);
} else if (application.sourceType === "gitea") {
command += await getGiteaCloneCommand(application, deployment.logPath);
} else if (application.sourceType === "git") {
command += await getCustomGitCloneCommand(
application,
deployment.logPath,
);
} else if (application.sourceType === "docker") {
command += await buildRemoteDocker(application, deployment.logPath);
}
if (application.sourceType !== "docker") {
command += getBuildCommand(application, deployment.logPath);
}
await execAsyncRemote(application.serverId, command);
await mechanizeDockerContainer(application);
await execAsyncRemote(application.serverId, commandWithLog);
} else {
await execAsync(commandWithLog);
}
await mechanizeDockerContainer(application);
await updateDeploymentStatus(deployment.deploymentId, "done");
await updateApplicationStatus(applicationId, "done");
if (application.rollbackActive) {
const tagImage =
application.sourceType === "docker"
? application.dockerImage
: application.appName;
await createRollback({
appName: tagImage || "",
deploymentId: deployment.deploymentId,
});
}
await sendBuildSuccessNotifications({
projectName: application.environment.project.name,
applicationName: application.name,
applicationType: "application",
buildLink,
organizationId: application.environment.project.organizationId,
domains: application.domains,
});
} catch (error) {
const errorMessage = error instanceof Error ? error.message : String(error);
const encodedContent = encodeBase64(errorMessage);
await execAsyncRemote(
application.serverId,
`
echo "\n\n===================================EXTRA LOGS============================================" >> ${deployment.logPath};
echo "Error occurred ❌, check the logs for details." >> ${deployment.logPath};
echo "${encodedContent}" | base64 -d >> "${deployment.logPath}";`,
);
await updateDeploymentStatus(deployment.deploymentId, "error");
await updateApplicationStatus(applicationId, "error");
await sendBuildErrorNotifications({
projectName: application.environment.project.name,
applicationName: application.name,
applicationType: "application",
errorMessage: `Please check the logs for details: ${errorMessage}`,
buildLink,
organizationId: application.environment.project.organizationId,
});
throw error;
}
@@ -475,14 +372,22 @@ export const deployPreviewApplication = async ({
application.buildArgs = `${application.previewBuildArgs}\nDOKPLOY_DEPLOY_URL=${previewDeployment?.domain?.host}`;
application.buildSecrets = `${application.previewBuildSecrets}\nDOKPLOY_DEPLOY_URL=${previewDeployment?.domain?.host}`;
let command = "set -e;";
if (application.sourceType === "github") {
await cloneGithubRepository({
command += await cloneGithubRepository({
...application,
appName: previewDeployment.appName,
branch: previewDeployment.branch,
logPath: deployment.logPath,
});
await buildApplication(application, deployment.logPath);
command += getBuildCommand(application);
const commandWithLog = `(${command}) >> ${deployment.logPath} 2>&1`;
if (application.serverId) {
await execAsyncRemote(application.serverId, commandWithLog);
} else {
await execAsync(commandWithLog);
}
await mechanizeDockerContainer(application);
}
const successComment = getIssueComment(
application.name,
@@ -513,170 +418,10 @@ export const deployPreviewApplication = async ({
return true;
};
export const deployRemotePreviewApplication = async ({
applicationId,
titleLog = "Preview Deployment",
descriptionLog = "",
previewDeploymentId,
}: {
applicationId: string;
titleLog: string;
descriptionLog: string;
previewDeploymentId: string;
}) => {
const application = await findApplicationById(applicationId);
const deployment = await createDeploymentPreview({
title: titleLog,
description: descriptionLog,
previewDeploymentId: previewDeploymentId,
});
const previewDeployment =
await findPreviewDeploymentById(previewDeploymentId);
await updatePreviewDeployment(previewDeploymentId, {
createdAt: new Date().toISOString(),
});
const previewDomain = getDomainHost(previewDeployment?.domain as Domain);
const issueParams = {
owner: application?.owner || "",
repository: application?.repository || "",
issue_number: previewDeployment.pullRequestNumber,
comment_id: Number.parseInt(previewDeployment.pullRequestCommentId),
githubId: application?.githubId || "",
};
try {
const commentExists = await issueCommentExists({
...issueParams,
});
if (!commentExists) {
const result = await createPreviewDeploymentComment({
...issueParams,
previewDomain,
appName: previewDeployment.appName,
githubId: application?.githubId || "",
previewDeploymentId,
});
if (!result) {
throw new TRPCError({
code: "NOT_FOUND",
message: "Pull request comment not found",
});
}
issueParams.comment_id = Number.parseInt(result?.pullRequestCommentId);
}
const buildingComment = getIssueComment(
application.name,
"running",
previewDomain,
);
await updateIssueComment({
...issueParams,
body: `### Dokploy Preview Deployment\n\n${buildingComment}`,
});
application.appName = previewDeployment.appName;
application.env = `${application.previewEnv}\nDOKPLOY_DEPLOY_URL=${previewDeployment?.domain?.host}`;
application.buildArgs = `${application.previewBuildArgs}\nDOKPLOY_DEPLOY_URL=${previewDeployment?.domain?.host}`;
application.buildSecrets = `${application.previewBuildSecrets}\nDOKPLOY_DEPLOY_URL=${previewDeployment?.domain?.host}`;
if (application.serverId) {
let command = "set -e;";
if (application.sourceType === "github") {
command += await getGithubCloneCommand({
...application,
appName: previewDeployment.appName,
branch: previewDeployment.branch,
serverId: application.serverId,
logPath: deployment.logPath,
});
}
command += getBuildCommand(application, deployment.logPath);
await execAsyncRemote(application.serverId, command);
await mechanizeDockerContainer(application);
}
const successComment = getIssueComment(
application.name,
"success",
previewDomain,
);
await updateIssueComment({
...issueParams,
body: `### Dokploy Preview Deployment\n\n${successComment}`,
});
await updateDeploymentStatus(deployment.deploymentId, "done");
await updatePreviewDeployment(previewDeploymentId, {
previewStatus: "done",
});
} catch (error) {
const comment = getIssueComment(application.name, "error", previewDomain);
await updateIssueComment({
...issueParams,
body: `### Dokploy Preview Deployment\n\n${comment}`,
});
await updateDeploymentStatus(deployment.deploymentId, "error");
await updatePreviewDeployment(previewDeploymentId, {
previewStatus: "error",
});
throw error;
}
return true;
};
export const rebuildRemoteApplication = async ({
applicationId,
titleLog = "Rebuild deployment",
descriptionLog = "",
}: {
applicationId: string;
titleLog: string;
descriptionLog: string;
}) => {
const application = await findApplicationById(applicationId);
const deployment = await createDeployment({
applicationId: applicationId,
title: titleLog,
description: descriptionLog,
});
try {
if (application.serverId) {
if (application.sourceType !== "docker") {
let command = "set -e;";
command += getBuildCommand(application, deployment.logPath);
await execAsyncRemote(application.serverId, command);
}
await mechanizeDockerContainer(application);
}
await updateDeploymentStatus(deployment.deploymentId, "done");
await updateApplicationStatus(applicationId, "done");
} catch (error) {
// @ts-ignore
const encodedContent = encodeBase64(error?.message);
await execAsyncRemote(
application.serverId,
`
echo "\n\n===================================EXTRA LOGS============================================" >> ${deployment.logPath};
echo "Error occurred ❌, check the logs for details." >> ${deployment.logPath};
echo "${encodedContent}" | base64 -d >> "${deployment.logPath}";`,
);
await updateDeploymentStatus(deployment.deploymentId, "error");
await updateApplicationStatus(applicationId, "error");
throw error;
}
return true;
};
export const getApplicationStats = async (appName: string) => {
if (appName === "dokploy") {
return await getAdvancedStats(appName);
}
const filter = {
status: ["running"],
label: [`com.docker.swarm.service.name=${appName}`],

View File

@@ -616,6 +616,7 @@ const ARVANCLOUD_IP_RANGES = [
"37.32.18.0/27",
"37.32.19.0/27",
"185.215.232.0/22",
"178.131.120.48/28",
];
const CDN_PROVIDERS: CDNProvider[] = [

View File

@@ -7,14 +7,10 @@ import {
cleanAppName,
compose,
} from "@dokploy/server/db/schema";
import {
buildCompose,
getBuildComposeCommand,
} from "@dokploy/server/utils/builders/compose";
import { getBuildComposeCommand } from "@dokploy/server/utils/builders/compose";
import { randomizeSpecificationFile } from "@dokploy/server/utils/docker/compose";
import {
cloneCompose,
cloneComposeRemote,
loadDockerCompose,
loadDockerComposeRemote,
} from "@dokploy/server/utils/docker/domain";
@@ -25,35 +21,23 @@ import {
execAsync,
execAsyncRemote,
} from "@dokploy/server/utils/process/execAsync";
import {
cloneBitbucketRepository,
getBitbucketCloneCommand,
} from "@dokploy/server/utils/providers/bitbucket";
import { cloneBitbucketRepository } from "@dokploy/server/utils/providers/bitbucket";
import {
cloneGitRepository,
getCustomGitCloneCommand,
getGitCommitInfo,
} from "@dokploy/server/utils/providers/git";
import {
cloneGiteaRepository,
getGiteaCloneCommand,
} from "@dokploy/server/utils/providers/gitea";
import {
cloneGithubRepository,
getGithubCloneCommand,
} from "@dokploy/server/utils/providers/github";
import {
cloneGitlabRepository,
getGitlabCloneCommand,
} from "@dokploy/server/utils/providers/gitlab";
import {
createComposeFile,
getCreateComposeFileCommand,
} from "@dokploy/server/utils/providers/raw";
import { cloneGiteaRepository } from "@dokploy/server/utils/providers/gitea";
import { cloneGithubRepository } from "@dokploy/server/utils/providers/github";
import { cloneGitlabRepository } from "@dokploy/server/utils/providers/gitlab";
import { getCreateComposeFileCommand } from "@dokploy/server/utils/providers/raw";
import { TRPCError } from "@trpc/server";
import { eq } from "drizzle-orm";
import { encodeBase64 } from "../utils/docker/utils";
import { getDokployUrl } from "./admin";
import { createDeploymentCompose, updateDeploymentStatus } from "./deployment";
import {
createDeploymentCompose,
updateDeployment,
updateDeploymentStatus,
} from "./deployment";
import { validUniqueServerAppName } from "./project";
export type Compose = typeof compose.$inferSelect;
@@ -163,10 +147,11 @@ export const loadServices = async (
const compose = await findComposeById(composeId);
if (type === "fetch") {
const command = await cloneCompose(compose);
if (compose.serverId) {
await cloneComposeRemote(compose);
await execAsyncRemote(compose.serverId, command);
} else {
await cloneCompose(compose);
await execAsync(command);
}
}
@@ -235,24 +220,41 @@ export const deployCompose = async ({
});
try {
const entity = {
...compose,
type: "compose" as const,
};
let command = "set -e;";
if (compose.sourceType === "github") {
await cloneGithubRepository({
...compose,
logPath: deployment.logPath,
type: "compose",
});
command += await cloneGithubRepository(entity);
} else if (compose.sourceType === "gitlab") {
await cloneGitlabRepository(compose, deployment.logPath, true);
command += await cloneGitlabRepository(entity);
} else if (compose.sourceType === "bitbucket") {
await cloneBitbucketRepository(compose, deployment.logPath, true);
command += await cloneBitbucketRepository(entity);
} else if (compose.sourceType === "git") {
await cloneGitRepository(compose, deployment.logPath, true);
command += await cloneGitRepository(entity);
} else if (compose.sourceType === "gitea") {
await cloneGiteaRepository(compose, deployment.logPath, true);
command += await cloneGiteaRepository(entity);
} else if (compose.sourceType === "raw") {
await createComposeFile(compose, deployment.logPath);
command += getCreateComposeFileCommand(entity);
}
await buildCompose(compose, deployment.logPath);
let commandWithLog = `(${command}) >> ${deployment.logPath} 2>&1`;
if (compose.serverId) {
await execAsyncRemote(compose.serverId, commandWithLog);
} else {
await execAsync(commandWithLog);
}
command = "set -e;";
command += await getBuildComposeCommand(entity);
commandWithLog = `(${command}) >> ${deployment.logPath} 2>&1`;
if (compose.serverId) {
await execAsyncRemote(compose.serverId, commandWithLog);
} else {
await execAsync(commandWithLog);
}
await updateDeploymentStatus(deployment.deploymentId, "done");
await updateCompose(composeId, {
composeStatus: "done",
@@ -281,6 +283,19 @@ export const deployCompose = async ({
organizationId: compose.environment.project.organizationId,
});
throw error;
} finally {
if (compose.sourceType !== "raw") {
const commitInfo = await getGitCommitInfo({
...compose,
type: "compose",
});
if (commitInfo) {
await updateDeployment(deployment.deploymentId, {
title: commitInfo.message,
description: `Commit: ${commitInfo.hash}`,
});
}
}
}
};
@@ -302,154 +317,23 @@ export const rebuildCompose = async ({
});
try {
let command = "set -e;";
if (compose.sourceType === "raw") {
await createComposeFile(compose, deployment.logPath);
command += getCreateComposeFileCommand(compose);
}
await buildCompose(compose, deployment.logPath);
await updateDeploymentStatus(deployment.deploymentId, "done");
await updateCompose(composeId, {
composeStatus: "done",
});
} catch (error) {
await updateDeploymentStatus(deployment.deploymentId, "error");
await updateCompose(composeId, {
composeStatus: "error",
});
throw error;
}
return true;
};
export const deployRemoteCompose = async ({
composeId,
titleLog = "Manual deployment",
descriptionLog = "",
}: {
composeId: string;
titleLog: string;
descriptionLog: string;
}) => {
const compose = await findComposeById(composeId);
const buildLink = `${await getDokployUrl()}/dashboard/project/${
compose.environment.projectId
}/environment/${compose.environmentId}/services/compose/${compose.composeId}?tab=deployments`;
const deployment = await createDeploymentCompose({
composeId: composeId,
title: titleLog,
description: descriptionLog,
});
try {
let commandWithLog = `(${command}) >> ${deployment.logPath} 2>&1`;
if (compose.serverId) {
let command = "set -e;";
if (compose.sourceType === "github") {
command += await getGithubCloneCommand({
...compose,
logPath: deployment.logPath,
type: "compose",
serverId: compose.serverId,
});
} else if (compose.sourceType === "gitlab") {
command += await getGitlabCloneCommand(
compose,
deployment.logPath,
true,
);
} else if (compose.sourceType === "bitbucket") {
command += await getBitbucketCloneCommand(
compose,
deployment.logPath,
true,
);
} else if (compose.sourceType === "git") {
command += await getCustomGitCloneCommand(
compose,
deployment.logPath,
true,
);
console.log(command);
} else if (compose.sourceType === "raw") {
command += getCreateComposeFileCommand(compose, deployment.logPath);
} else if (compose.sourceType === "gitea") {
command += await getGiteaCloneCommand(
compose,
deployment.logPath,
true,
);
}
await execAsyncRemote(compose.serverId, command);
await getBuildComposeCommand(compose, deployment.logPath);
}
await updateDeploymentStatus(deployment.deploymentId, "done");
await updateCompose(composeId, {
composeStatus: "done",
});
await sendBuildSuccessNotifications({
projectName: compose.environment.project.name,
applicationName: compose.name,
applicationType: "compose",
buildLink,
organizationId: compose.environment.project.organizationId,
domains: compose.domains,
});
} catch (error) {
// @ts-ignore
const encodedContent = encodeBase64(error?.message);
await execAsyncRemote(
compose.serverId,
`
echo "\n\n===================================EXTRA LOGS============================================" >> ${deployment.logPath};
echo "Error occurred ❌, check the logs for details." >> ${deployment.logPath};
echo "${encodedContent}" | base64 -d >> "${deployment.logPath}";`,
);
await updateDeploymentStatus(deployment.deploymentId, "error");
await updateCompose(composeId, {
composeStatus: "error",
});
await sendBuildErrorNotifications({
projectName: compose.environment.project.name,
applicationName: compose.name,
applicationType: "compose",
// @ts-ignore
errorMessage: error?.message || "Error building",
buildLink,
organizationId: compose.environment.project.organizationId,
});
throw error;
}
};
export const rebuildRemoteCompose = async ({
composeId,
titleLog = "Rebuild deployment",
descriptionLog = "",
}: {
composeId: string;
titleLog: string;
descriptionLog: string;
}) => {
const compose = await findComposeById(composeId);
const deployment = await createDeploymentCompose({
composeId: composeId,
title: titleLog,
description: descriptionLog,
});
try {
if (compose.sourceType === "raw") {
const command = getCreateComposeFileCommand(compose, deployment.logPath);
await execAsyncRemote(compose.serverId, command);
await execAsyncRemote(compose.serverId, commandWithLog);
} else {
await execAsync(commandWithLog);
}
command += await getBuildComposeCommand(compose);
commandWithLog = `(${command}) >> ${deployment.logPath} 2>&1`;
if (compose.serverId) {
await getBuildComposeCommand(compose, deployment.logPath);
await execAsyncRemote(compose.serverId, commandWithLog);
} else {
await execAsync(commandWithLog);
}
await updateDeploymentStatus(deployment.deploymentId, "done");
@@ -457,16 +341,6 @@ export const rebuildRemoteCompose = async ({
composeStatus: "done",
});
} catch (error) {
// @ts-ignore
const encodedContent = encodeBase64(error?.message);
await execAsyncRemote(
compose.serverId,
`
echo "\n\n===================================EXTRA LOGS============================================" >> ${deployment.logPath};
echo "Error occurred ❌, check the logs for details." >> ${deployment.logPath};
echo "${encodedContent}" | base64 -d >> "${deployment.logPath}";`,
);
await updateDeploymentStatus(deployment.deploymentId, "error");
await updateCompose(composeId, {
composeStatus: "error",

View File

@@ -74,20 +74,21 @@ export const createDeployment = async (
>,
) => {
const application = await findApplicationById(deployment.applicationId);
try {
await removeLastTenDeployments(
deployment.applicationId,
"application",
application.serverId,
);
const { LOGS_PATH } = paths(!!application.serverId);
const serverId = application.serverId;
const { LOGS_PATH } = paths(!!serverId);
const formattedDateTime = format(new Date(), "yyyy-MM-dd:HH:mm:ss");
const fileName = `${application.appName}-${formattedDateTime}.log`;
const logFilePath = path.join(LOGS_PATH, application.appName, fileName);
if (application.serverId) {
const server = await findServerById(application.serverId);
if (serverId) {
const server = await findServerById(serverId);
const command = `
mkdir -p ${LOGS_PATH}/${application.appName};
@@ -99,7 +100,7 @@ export const createDeployment = async (
await fsPromises.mkdir(path.join(LOGS_PATH, application.appName), {
recursive: true,
});
await fsPromises.writeFile(logFilePath, "Initializing deployment");
await fsPromises.writeFile(logFilePath, "Initializing deployment\n");
}
const deploymentCreate = await db
@@ -249,7 +250,7 @@ export const createDeploymentCompose = async (
const command = `
mkdir -p ${LOGS_PATH}/${compose.appName};
echo "Initializing deployment" >> ${logFilePath};
echo "Initializing deployment\n" >> ${logFilePath};
`;
await execAsyncRemote(server.serverId, command);
@@ -257,7 +258,7 @@ echo "Initializing deployment" >> ${logFilePath};
await fsPromises.mkdir(path.join(LOGS_PATH, compose.appName), {
recursive: true,
});
await fsPromises.writeFile(logFilePath, "Initializing deployment");
await fsPromises.writeFile(logFilePath, "Initializing deployment\n");
}
const deploymentCreate = await db

View File

@@ -34,13 +34,43 @@ export const findEnvironmentById = async (environmentId: string) => {
const environment = await db.query.environments.findFirst({
where: eq(environments.environmentId, environmentId),
with: {
applications: true,
mariadb: true,
mongo: true,
mysql: true,
postgres: true,
redis: true,
compose: true,
applications: {
with: {
deployments: true,
server: true,
},
},
mariadb: {
with: {
server: true,
},
},
mongo: {
with: {
server: true,
},
},
mysql: {
with: {
server: true,
},
},
postgres: {
with: {
server: true,
},
},
redis: {
with: {
server: true,
},
},
compose: {
with: {
deployments: true,
server: true,
},
},
project: true,
},
});

View File

@@ -1,10 +1,10 @@
import { db } from "@dokploy/server/db";
import { apikey, member, users_temp } from "@dokploy/server/db/schema";
import { apikey, member, user } from "@dokploy/server/db/schema";
import { TRPCError } from "@trpc/server";
import { and, eq } from "drizzle-orm";
import { auth } from "../lib/auth";
export type User = typeof users_temp.$inferSelect;
export type User = typeof user.$inferSelect;
export const addNewProject = async (
userId: string,
@@ -403,16 +403,16 @@ export const updateUser = async (userId: string, userData: Partial<User>) => {
}
}
const user = await db
.update(users_temp)
const userResult = await db
.update(user)
.set({
...userData,
})
.where(eq(users_temp.id, userId))
.where(eq(user.id, userId))
.returning()
.then((res) => res[0]);
return user;
return userResult;
};
export const createApiKey = async (

View File

@@ -20,7 +20,7 @@ export const TRAEFIK_PORT =
Number.parseInt(process.env.TRAEFIK_PORT!, 10) || 80;
export const TRAEFIK_HTTP3_PORT =
Number.parseInt(process.env.TRAEFIK_HTTP3_PORT!, 10) || 443;
export const TRAEFIK_VERSION = process.env.TRAEFIK_VERSION || "3.5.0";
export const TRAEFIK_VERSION = process.env.TRAEFIK_VERSION || "3.6.1";
export interface TraefikOptions {
env?: string[];

View File

@@ -37,7 +37,16 @@ export const generateRandomDomain = ({
const hash = randomBytes(3).toString("hex");
const slugIp = serverIp.replaceAll(".", "-").replaceAll(":", "-");
return `${projectName}-${hash}${slugIp === "" ? "" : `-${slugIp}`}.traefik.me`;
// Domain labels have a max length of 63 characters
// Reserve space for: hash (6) + separators (1-2) + ip section + dot + traefik.me (10)
// Approx: 6 + 2 + (variable ip length) + 11 = ~19-30 chars for other parts
const maxProjectNameLength = 40;
const truncatedProjectName =
projectName.length > maxProjectNameLength
? projectName.substring(0, maxProjectNameLength)
: projectName;
return `${truncatedProjectName}-${hash}${slugIp === "" ? "" : `-${slugIp}`}.traefik.me`;
};
export const generateHash = (length = 8): string => {

View File

@@ -1,117 +1,28 @@
import {
createWriteStream,
existsSync,
mkdirSync,
writeFileSync,
} from "node:fs";
import { dirname, join } from "node:path";
import { paths } from "@dokploy/server/constants";
import type { InferResultType } from "@dokploy/server/types/with";
import boxen from "boxen";
import {
writeDomainsToCompose,
writeDomainsToComposeRemote,
} from "../docker/domain";
import { writeDomainsToComposeRemote } from "../docker/domain";
import {
encodeBase64,
getEnviromentVariablesObject,
prepareEnvironmentVariables,
} from "../docker/utils";
import { execAsync, execAsyncRemote } from "../process/execAsync";
import { spawnAsync } from "../process/spawnAsync";
export type ComposeNested = InferResultType<
"compose",
{ environment: { with: { project: true } }; mounts: true; domains: true }
>;
export const buildCompose = async (compose: ComposeNested, logPath: string) => {
const writeStream = createWriteStream(logPath, { flags: "a" });
const { sourceType, appName, mounts, composeType, domains } = compose;
try {
const { COMPOSE_PATH } = paths();
const command = createCommand(compose);
await writeDomainsToCompose(compose, domains);
createEnvFile(compose);
if (compose.isolatedDeployment) {
await execAsync(
`docker network inspect ${compose.appName} >/dev/null 2>&1 || docker network create ${composeType === "stack" ? "--driver overlay" : ""} --attachable ${compose.appName}`,
);
}
const logContent = `
App Name: ${appName}
Build Compose 🐳
Detected: ${mounts.length} mounts 📂
Command: docker ${command}
Source Type: docker ${sourceType}
Compose Type: ${composeType}`;
const logBox = boxen(logContent, {
padding: {
left: 1,
right: 1,
bottom: 1,
},
width: 80,
borderStyle: "double",
});
writeStream.write(`\n${logBox}\n`);
const projectPath = join(COMPOSE_PATH, compose.appName, "code");
await spawnAsync(
"docker",
[...command.split(" ")],
(data) => {
if (writeStream.writable) {
writeStream.write(data.toString());
}
},
{
cwd: projectPath,
env: {
NODE_ENV: process.env.NODE_ENV,
PATH: process.env.PATH,
...(composeType === "stack" && {
...getEnviromentVariablesObject(
compose.env,
compose.environment.project.env,
),
}),
},
},
);
if (compose.isolatedDeployment) {
await execAsync(
`docker network connect ${compose.appName} $(docker ps --filter "name=dokploy-traefik" -q) >/dev/null 2>&1`,
).catch(() => {});
}
writeStream.write("Docker Compose Deployed: ✅");
} catch (error) {
writeStream.write(`Error ❌ ${(error as Error).message}`);
throw error;
} finally {
writeStream.end();
}
};
export const getBuildComposeCommand = async (
compose: ComposeNested,
logPath: string,
) => {
const { COMPOSE_PATH } = paths(true);
export const getBuildComposeCommand = async (compose: ComposeNested) => {
const { COMPOSE_PATH } = paths(!!compose.serverId);
const { sourceType, appName, mounts, composeType, domains } = compose;
const command = createCommand(compose);
const envCommand = getCreateEnvFileCommand(compose);
const projectPath = join(COMPOSE_PATH, compose.appName, "code");
const exportEnvCommand = getExportEnvCommand(compose);
const newCompose = await writeDomainsToComposeRemote(
compose,
domains,
logPath,
);
const newCompose = await writeDomainsToComposeRemote(compose, domains);
const logContent = `
App Name: ${appName}
Build Compose 🐳
@@ -133,7 +44,7 @@ Compose Type: ${composeType} ✅`;
const bashCommand = `
set -e
{
echo "${logBox}" >> "${logPath}"
echo "${logBox}";
${newCompose}
@@ -143,17 +54,18 @@ Compose Type: ${composeType} ✅`;
${exportEnvCommand}
${compose.isolatedDeployment ? `docker network inspect ${compose.appName} >/dev/null 2>&1 || docker network create --attachable ${compose.appName}` : ""}
docker ${command.split(" ").join(" ")} >> "${logPath}" 2>&1 || { echo "Error: ❌ Docker command failed" >> "${logPath}"; exit 1; }
docker ${command.split(" ").join(" ")} 2>&1 || { echo "Error: ❌ Docker command failed"; exit 1; }
${compose.isolatedDeployment ? `docker network connect ${compose.appName} $(docker ps --filter "name=dokploy-traefik" -q) >/dev/null 2>&1` : ""}
echo "Docker Compose Deployed: ✅" >> "${logPath}"
echo "Docker Compose Deployed: ✅";
} || {
echo "Error: ❌ Script execution failed" >> "${logPath}"
echo "Error: ❌ Script execution failed";
exit 1
}
`;
return await execAsyncRemote(compose.serverId, bashCommand);
return bashCommand;
// return await execAsyncRemote(compose.serverId, bashCommand);
};
const sanitizeCommand = (command: string) => {
@@ -185,38 +97,8 @@ export const createCommand = (compose: ComposeNested) => {
return command;
};
const createEnvFile = (compose: ComposeNested) => {
const { COMPOSE_PATH } = paths();
const { env, composePath, appName } = compose;
const composeFilePath =
join(COMPOSE_PATH, appName, "code", composePath) ||
join(COMPOSE_PATH, appName, "code", "docker-compose.yml");
const envFilePath = join(dirname(composeFilePath), ".env");
let envContent = `APP_NAME=${appName}\n`;
envContent += env || "";
if (!envContent.includes("DOCKER_CONFIG")) {
envContent += "\nDOCKER_CONFIG=/root/.docker";
}
if (compose.randomize) {
envContent += `\nCOMPOSE_PREFIX=${compose.suffix}`;
}
const envFileContent = prepareEnvironmentVariables(
envContent,
compose.environment.project.env,
compose.environment.env,
).join("\n");
if (!existsSync(dirname(envFilePath))) {
mkdirSync(dirname(envFilePath), { recursive: true });
}
writeFileSync(envFilePath, envFileContent);
};
export const getCreateEnvFileCommand = (compose: ComposeNested) => {
const { COMPOSE_PATH } = paths(true);
const { COMPOSE_PATH } = paths(!!compose.serverId);
const { env, composePath, appName } = compose;
const composeFilePath =
join(COMPOSE_PATH, appName, "code", composePath) ||

View File

@@ -1,4 +1,3 @@
import type { WriteStream } from "node:fs";
import {
getEnviromentVariablesObject,
prepareEnvironmentVariables,
@@ -7,103 +6,10 @@ import {
getBuildAppDirectory,
getDockerContextPath,
} from "../filesystem/directory";
import { spawnAsync } from "../process/spawnAsync";
import type { ApplicationNested } from ".";
import { createEnvFile, createEnvFileCommand } from "./utils";
import { createEnvFileCommand } from "./utils";
export const buildCustomDocker = async (
application: ApplicationNested,
writeStream: WriteStream,
) => {
const {
appName,
env,
publishDirectory,
buildArgs,
buildSecrets,
dockerBuildStage,
cleanCache,
} = application;
const dockerFilePath = getBuildAppDirectory(application);
try {
const image = `${appName}`;
const defaultContextPath =
dockerFilePath.substring(0, dockerFilePath.lastIndexOf("/") + 1) || ".";
const dockerContextPath = getDockerContextPath(application);
const commandArgs = ["build", "-t", image, "-f", dockerFilePath, "."];
if (cleanCache) {
commandArgs.push("--no-cache");
}
if (dockerBuildStage) {
commandArgs.push("--target", dockerBuildStage);
}
const args = prepareEnvironmentVariables(
buildArgs,
application.environment.project.env,
application.environment.env,
);
for (const arg of args) {
commandArgs.push("--build-arg", arg);
}
const secrets = getEnviromentVariablesObject(
buildSecrets,
application.environment.project.env,
application.environment.env,
);
for (const key in secrets) {
// Although buildx is smart enough to know we may be referring to an environment variable name,
// we still make sure it doesn't fall back to type=file.
// See: https://docs.docker.com/reference/cli/docker/buildx/build/#secret
commandArgs.push("--secret", `type=env,id=${key}`);
}
/*
Do not generate an environment file when publishDirectory is specified,
as it could be publicly exposed.
*/
if (!publishDirectory) {
createEnvFile(
dockerFilePath,
env,
application.environment.project.env,
application.environment.env,
);
}
await spawnAsync(
"docker",
commandArgs,
(data) => {
if (writeStream.writable) {
writeStream.write(data);
}
},
{
cwd: dockerContextPath || defaultContextPath,
env: {
...process.env,
...secrets,
},
},
);
} catch (error) {
throw error;
}
};
export const getDockerCommand = (
application: ApplicationNested,
logPath: string,
) => {
export const getDockerCommand = (application: ApplicationNested) => {
const {
appName,
env,
@@ -176,17 +82,17 @@ export const getDockerCommand = (
}
command += `
echo "Building ${appName}" >> ${logPath};
cd ${dockerContextPath} >> ${logPath} 2>> ${logPath} || {
echo "❌ The path ${dockerContextPath} does not exist" >> ${logPath};
echo "Building ${appName}" ;
cd ${dockerContextPath} || {
echo "❌ The path ${dockerContextPath} does not exist" ;
exit 1;
}
${joinedSecrets} docker ${commandArgs.join(" ")} >> ${logPath} 2>> ${logPath} || {
echo "❌ Docker build failed" >> ${logPath};
${joinedSecrets} docker ${commandArgs.join(" ")} || {
echo "❌ Docker build failed" ;
exit 1;
}
echo "✅ Docker build completed." >> ${logPath};
echo "✅ Docker build completed." ;
`;
return command;

View File

@@ -1,54 +1,8 @@
import type { WriteStream } from "node:fs";
import { prepareEnvironmentVariables } from "../docker/utils";
import { getBuildAppDirectory } from "../filesystem/directory";
import { spawnAsync } from "../process/spawnAsync";
import type { ApplicationNested } from ".";
// TODO: integrate in the vps sudo chown -R $(whoami) ~/.docker
export const buildHeroku = async (
application: ApplicationNested,
writeStream: WriteStream,
) => {
const { env, appName, cleanCache } = application;
const buildAppDirectory = getBuildAppDirectory(application);
const envVariables = prepareEnvironmentVariables(
env,
application.environment.project.env,
application.environment.env,
);
try {
const args = [
"build",
appName,
"--path",
buildAppDirectory,
"--builder",
`heroku/builder:${application.herokuVersion || "24"}`,
];
for (const env of envVariables) {
args.push("--env", env);
}
if (cleanCache) {
args.push("--clear-cache");
}
await spawnAsync("pack", args, (data) => {
if (writeStream.writable) {
writeStream.write(data);
}
});
return true;
} catch (e) {
throw e;
}
};
export const getHerokuCommand = (
application: ApplicationNested,
logPath: string,
) => {
export const getHerokuCommand = (application: ApplicationNested) => {
const { env, appName, cleanCache } = application;
const buildAppDirectory = getBuildAppDirectory(application);
@@ -77,12 +31,12 @@ export const getHerokuCommand = (
const command = `pack ${args.join(" ")}`;
const bashCommand = `
echo "Starting heroku build..." >> ${logPath};
${command} >> ${logPath} 2>> ${logPath} || {
echo "❌ Heroku build failed" >> ${logPath};
echo "Starting heroku build..." ;
${command} || {
echo "❌ Heroku build failed" ;
exit 1;
}
echo "✅ Heroku build completed." >> ${logPath};
echo "✅ Heroku build completed." ;
`;
return bashCommand;

View File

@@ -1,7 +1,6 @@
import { createWriteStream } from "node:fs";
import type { InferResultType } from "@dokploy/server/types/with";
import type { CreateServiceOptions } from "dockerode";
import { uploadImage, uploadImageRemoteCommand } from "../cluster/upload";
import { uploadImageRemoteCommand } from "../cluster/upload";
import {
calculateResources,
generateBindMounts,
@@ -11,12 +10,12 @@ import {
prepareEnvironmentVariables,
} from "../docker/utils";
import { getRemoteDocker } from "../servers/remote-docker";
import { buildCustomDocker, getDockerCommand } from "./docker-file";
import { buildHeroku, getHerokuCommand } from "./heroku";
import { buildNixpacks, getNixpacksCommand } from "./nixpacks";
import { buildPaketo, getPaketoCommand } from "./paketo";
import { buildRailpack, getRailpackCommand } from "./railpack";
import { buildStatic, getStaticCommand } from "./static";
import { getDockerCommand } from "./docker-file";
import { getHerokuCommand } from "./heroku";
import { getNixpacksCommand } from "./nixpacks";
import { getPaketoCommand } from "./paketo";
import { getRailpackCommand } from "./railpack";
import { getStaticCommand } from "./static";
// NIXPACKS codeDirectory = where is the path of the code directory
// HEROKU codeDirectory = where is the path of the code directory
@@ -34,76 +33,35 @@ export type ApplicationNested = InferResultType<
}
>;
export const buildApplication = async (
application: ApplicationNested,
logPath: string,
) => {
const writeStream = createWriteStream(logPath, { flags: "a" });
const { buildType, sourceType } = application;
try {
writeStream.write(
`\nBuild ${buildType}: ✅\nSource Type: ${sourceType}: ✅\n`,
);
console.log(`Build ${buildType}: ✅`);
if (buildType === "nixpacks") {
await buildNixpacks(application, writeStream);
} else if (buildType === "heroku_buildpacks") {
await buildHeroku(application, writeStream);
} else if (buildType === "paketo_buildpacks") {
await buildPaketo(application, writeStream);
} else if (buildType === "dockerfile") {
await buildCustomDocker(application, writeStream);
} else if (buildType === "static") {
await buildStatic(application, writeStream);
} else if (buildType === "railpack") {
await buildRailpack(application, writeStream);
}
if (application.registryId) {
await uploadImage(application, writeStream);
}
await mechanizeDockerContainer(application);
writeStream.write("Docker Deployed: ✅");
} catch (error) {
if (error instanceof Error) {
writeStream.write(`Error ❌\n${error?.message}`);
} else {
writeStream.write("Error ❌");
}
throw error;
} finally {
writeStream.end();
}
};
export const getBuildCommand = (
application: ApplicationNested,
logPath: string,
) => {
export const getBuildCommand = (application: ApplicationNested) => {
let command = "";
const { buildType, registry } = application;
if (application.sourceType === "docker") {
return "";
}
switch (buildType) {
case "nixpacks":
command = getNixpacksCommand(application, logPath);
command = getNixpacksCommand(application);
break;
case "heroku_buildpacks":
command = getHerokuCommand(application, logPath);
command = getHerokuCommand(application);
break;
case "paketo_buildpacks":
command = getPaketoCommand(application, logPath);
command = getPaketoCommand(application);
break;
case "static":
command = getStaticCommand(application, logPath);
command = getStaticCommand(application);
break;
case "dockerfile":
command = getDockerCommand(application, logPath);
command = getDockerCommand(application);
break;
case "railpack":
command = getRailpackCommand(application, logPath);
command = getRailpackCommand(application);
break;
}
if (registry) {
command += uploadImageRemoteCommand(application, logPath);
command += uploadImageRemoteCommand(application);
}
return command;
@@ -143,6 +101,7 @@ export const mechanizeDockerContainer = async (
UpdateConfig,
Networks,
StopGracePeriod,
EndpointSpec,
} = generateConfigContainer(application);
const bindsMount = generateBindMounts(mounts);
@@ -183,14 +142,16 @@ export const mechanizeDockerContainer = async (
},
Mode,
RollbackConfig,
EndpointSpec: {
Ports: ports.map((port) => ({
PublishMode: port.publishMode,
Protocol: port.protocol,
TargetPort: port.targetPort,
PublishedPort: port.publishedPort,
})),
},
EndpointSpec: EndpointSpec
? EndpointSpec
: {
Ports: ports.map((port) => ({
PublishMode: port.publishMode,
Protocol: port.protocol,
TargetPort: port.targetPort,
PublishedPort: port.publishedPort,
})),
},
UpdateConfig,
...(StopGracePeriod !== undefined &&
StopGracePeriod !== null && { StopGracePeriod }),

View File

@@ -1,101 +1,11 @@
import { existsSync, mkdirSync, type WriteStream } from "node:fs";
import path from "node:path";
import {
buildStatic,
getStaticCommand,
} from "@dokploy/server/utils/builders/static";
import { getStaticCommand } from "@dokploy/server/utils/builders/static";
import { nanoid } from "nanoid";
import { prepareEnvironmentVariables } from "../docker/utils";
import { getBuildAppDirectory } from "../filesystem/directory";
import { spawnAsync } from "../process/spawnAsync";
import type { ApplicationNested } from ".";
export const buildNixpacks = async (
application: ApplicationNested,
writeStream: WriteStream,
) => {
const { env, appName, publishDirectory, cleanCache } = application;
const buildAppDirectory = getBuildAppDirectory(application);
const buildContainerId = `${appName}-${nanoid(10)}`;
const envVariables = prepareEnvironmentVariables(
env,
application.environment.project.env,
application.environment.env,
);
const writeToStream = (data: string) => {
if (writeStream.writable) {
writeStream.write(data);
}
};
try {
const args = ["build", buildAppDirectory, "--name", appName];
if (cleanCache) {
args.push("--no-cache");
}
for (const env of envVariables) {
args.push("--env", env);
}
if (publishDirectory) {
/* No need for any start command, since we'll use nginx later on */
args.push("--no-error-without-start");
}
await spawnAsync("nixpacks", args, writeToStream);
/*
Run the container with the image created by nixpacks,
and copy the artifacts on the host filesystem.
Then, remove the container and create a static build.
*/
if (publishDirectory) {
await spawnAsync(
"docker",
["create", "--name", buildContainerId, appName],
writeToStream,
);
const localPath = path.join(buildAppDirectory, publishDirectory);
if (!existsSync(path.dirname(localPath))) {
mkdirSync(path.dirname(localPath), { recursive: true });
}
// https://docs.docker.com/reference/cli/docker/container/cp/
const isDirectory =
publishDirectory.endsWith("/") || !path.extname(publishDirectory);
await spawnAsync(
"docker",
[
"cp",
`${buildContainerId}:/app/${publishDirectory}${isDirectory ? "/." : ""}`,
localPath,
],
writeToStream,
);
await spawnAsync("docker", ["rm", buildContainerId], writeToStream);
await buildStatic(application, writeStream);
}
return true;
} catch (e) {
await spawnAsync("docker", ["rm", buildContainerId], writeToStream);
throw e;
}
};
export const getNixpacksCommand = (
application: ApplicationNested,
logPath: string,
) => {
export const getNixpacksCommand = (application: ApplicationNested) => {
const { env, appName, publishDirectory, cleanCache } = application;
const buildAppDirectory = getBuildAppDirectory(application);
@@ -122,12 +32,12 @@ export const getNixpacksCommand = (
}
const command = `nixpacks ${args.join(" ")}`;
let bashCommand = `
echo "Starting nixpacks build..." >> ${logPath};
${command} >> ${logPath} 2>> ${logPath} || {
echo "❌ Nixpacks build failed" >> ${logPath};
exit 1;
}
echo "✅ Nixpacks build completed." >> ${logPath};
echo "Starting nixpacks build..." ;
${command} || {
echo "❌ Nixpacks build failed" ;
exit 1;
}
echo "✅ Nixpacks build completed." ;
`;
/*
@@ -141,16 +51,16 @@ echo "✅ Nixpacks build completed." >> ${logPath};
publishDirectory.endsWith("/") || !path.extname(publishDirectory);
bashCommand += `
docker create --name ${buildContainerId} ${appName}
mkdir -p ${localPath}
docker cp ${buildContainerId}:/app/${publishDirectory}${isDirectory ? "/." : ""} ${path.join(buildAppDirectory, publishDirectory)} >> ${logPath} 2>> ${logPath} || {
docker create --name ${buildContainerId} ${appName}
mkdir -p ${localPath}
docker cp ${buildContainerId}:/app/${publishDirectory}${isDirectory ? "/." : ""} ${path.join(buildAppDirectory, publishDirectory)} || {
docker rm ${buildContainerId}
echo "❌ Copying ${publishDirectory} to ${path.join(buildAppDirectory, publishDirectory)} failed" ;
exit 1;
}
docker rm ${buildContainerId}
echo "❌ Copying ${publishDirectory} to ${path.join(buildAppDirectory, publishDirectory)} failed" >> ${logPath};
exit 1;
}
docker rm ${buildContainerId}
${getStaticCommand(application, logPath)}
`;
${getStaticCommand(application)}
`;
}
return bashCommand;

View File

@@ -1,53 +1,8 @@
import type { WriteStream } from "node:fs";
import { prepareEnvironmentVariables } from "../docker/utils";
import { getBuildAppDirectory } from "../filesystem/directory";
import { spawnAsync } from "../process/spawnAsync";
import type { ApplicationNested } from ".";
export const buildPaketo = async (
application: ApplicationNested,
writeStream: WriteStream,
) => {
const { env, appName, cleanCache } = application;
const buildAppDirectory = getBuildAppDirectory(application);
const envVariables = prepareEnvironmentVariables(
env,
application.environment.project.env,
application.environment.env,
);
try {
const args = [
"build",
appName,
"--path",
buildAppDirectory,
"--builder",
"paketobuildpacks/builder-jammy-full",
];
if (cleanCache) {
args.push("--clear-cache");
}
for (const env of envVariables) {
args.push("--env", env);
}
await spawnAsync("pack", args, (data) => {
if (writeStream.writable) {
writeStream.write(data);
}
});
return true;
} catch (e) {
throw e;
}
};
export const getPaketoCommand = (
application: ApplicationNested,
logPath: string,
) => {
export const getPaketoCommand = (application: ApplicationNested) => {
const { env, appName, cleanCache } = application;
const buildAppDirectory = getBuildAppDirectory(application);
@@ -76,12 +31,12 @@ export const getPaketoCommand = (
const command = `pack ${args.join(" ")}`;
const bashCommand = `
echo "Starting Paketo build..." >> ${logPath};
${command} >> ${logPath} 2>> ${logPath} || {
echo "❌ Paketo build failed" >> ${logPath};
echo "Starting Paketo build..." ;
${command} || {
echo "❌ Paketo build failed" ;
exit 1;
}
echo "✅ Paketo build completed." >> ${logPath};
echo "✅ Paketo build completed." ;
`;
return bashCommand;

View File

@@ -1,13 +1,10 @@
import { createHash } from "node:crypto";
import type { WriteStream } from "node:fs";
import { nanoid } from "nanoid";
import {
parseEnvironmentKeyValuePair,
prepareEnvironmentVariables,
} from "../docker/utils";
import { getBuildAppDirectory } from "../filesystem/directory";
import { execAsync } from "../process/execAsync";
import { spawnAsync } from "../process/spawnAsync";
import type { ApplicationNested } from ".";
const calculateSecretsHash = (envVariables: string[]): string => {
@@ -18,108 +15,7 @@ const calculateSecretsHash = (envVariables: string[]): string => {
return hash.digest("hex");
};
export const buildRailpack = async (
application: ApplicationNested,
writeStream: WriteStream,
) => {
const { env, appName, cleanCache } = application;
const buildAppDirectory = getBuildAppDirectory(application);
const envVariables = prepareEnvironmentVariables(
env,
application.environment.project.env,
application.environment.env,
);
try {
await execAsync(
"docker buildx create --use --name builder-containerd --driver docker-container || true",
);
await execAsync("docker buildx use builder-containerd");
// First prepare the build plan and info
const prepareArgs = [
"prepare",
buildAppDirectory,
"--plan-out",
`${buildAppDirectory}/railpack-plan.json`,
"--info-out",
`${buildAppDirectory}/railpack-info.json`,
];
// Add environment variables to prepare command
for (const env of envVariables) {
prepareArgs.push("--env", env);
}
// Run prepare command
await spawnAsync("railpack", prepareArgs, (data) => {
if (writeStream.writable) {
writeStream.write(data);
}
});
// Calculate secrets hash for layer invalidation
const secretsHash = calculateSecretsHash(envVariables);
// Build with BuildKit using the Railpack frontend
const cacheKey = cleanCache ? nanoid(10) : undefined;
const buildArgs = [
"buildx",
"build",
...(cacheKey
? [
"--build-arg",
`secrets-hash=${secretsHash}`,
"--build-arg",
`cache-key=${cacheKey}`,
]
: []),
"--build-arg",
`BUILDKIT_SYNTAX=ghcr.io/railwayapp/railpack-frontend:v${application.railpackVersion}`,
"-f",
`${buildAppDirectory}/railpack-plan.json`,
"--output",
`type=docker,name=${appName}`,
];
// Add secrets properly formatted
const env: { [key: string]: string } = {};
for (const pair of envVariables) {
const [key, value] = parseEnvironmentKeyValuePair(pair);
if (key && value) {
buildArgs.push("--secret", `id=${key},env=${key}`);
env[key] = value;
}
}
buildArgs.push(buildAppDirectory);
await spawnAsync(
"docker",
buildArgs,
(data) => {
if (writeStream.writable) {
writeStream.write(data);
}
},
{
env: { ...process.env, ...env },
},
);
return true;
} catch (e) {
throw e;
} finally {
await execAsync("docker buildx rm builder-containerd");
}
};
export const getRailpackCommand = (
application: ApplicationNested,
logPath: string,
) => {
export const getRailpackCommand = (application: ApplicationNested) => {
const { env, appName, cleanCache } = application;
const buildAppDirectory = getBuildAppDirectory(application);
const envVariables = prepareEnvironmentVariables(
@@ -179,26 +75,29 @@ export const getRailpackCommand = (
buildArgs.push(buildAppDirectory);
const bashCommand = `
# Ensure we have a builder with containerd
docker buildx create --use --name builder-containerd --driver docker-container || true
docker buildx use builder-containerd
echo "Preparing Railpack build plan..." >> "${logPath}";
railpack ${prepareArgs.join(" ")} >> ${logPath} 2>> ${logPath} || {
echo "❌ Railpack prepare failed" >> ${logPath};
echo "Preparing Railpack build plan..." ;
railpack ${prepareArgs.join(" ")} || {
echo "❌ Railpack prepare failed" ;
docker buildx rm builder-containerd || true
exit 1;
}
echo "✅ Railpack prepare completed." >> ${logPath};
echo "✅ Railpack prepare completed." ;
echo "Building with Railpack frontend..." >> "${logPath}";
echo "Building with Railpack frontend..." ;
# Export environment variables for secrets
${exportEnvs.join("\n")}
docker ${buildArgs.join(" ")} >> ${logPath} 2>> ${logPath} || {
echo "❌ Railpack build failed" >> ${logPath};
docker ${buildArgs.join(" ")} || {
echo "❌ Railpack build failed" ;
docker buildx rm builder-containerd || true
exit 1;
}
echo "✅ Railpack build completed." >> ${logPath};
docker buildx rm builder-containerd
echo "✅ Railpack build completed." ;
docker buildx rm builder-containerd || true
`;
return bashCommand;

View File

@@ -1,9 +1,5 @@
import type { WriteStream } from "node:fs";
import {
buildCustomDocker,
getDockerCommand,
} from "@dokploy/server/utils/builders/docker-file";
import { createFile, getCreateFileCommand } from "../docker/utils";
import { getDockerCommand } from "@dokploy/server/utils/builders/docker-file";
import { getCreateFileCommand } from "../docker/utils";
import { getBuildAppDirectory } from "../filesystem/directory";
import type { ApplicationNested } from ".";
@@ -32,81 +28,40 @@ http {
}
`;
export const buildStatic = async (
application: ApplicationNested,
writeStream: WriteStream,
) => {
export const getStaticCommand = (application: ApplicationNested) => {
const { publishDirectory, isStaticSpa } = application;
const buildAppDirectory = getBuildAppDirectory(application);
try {
if (isStaticSpa) {
createFile(buildAppDirectory, "nginx.conf", nginxSpaConfig);
}
createFile(
let command = "";
if (isStaticSpa) {
command += getCreateFileCommand(
buildAppDirectory,
".dockerignore",
[".git", ".env", "Dockerfile", ".dockerignore"].join("\n"),
"nginx.conf",
nginxSpaConfig,
);
createFile(
buildAppDirectory,
"Dockerfile",
[
"FROM nginx:alpine",
"WORKDIR /usr/share/nginx/html/",
isStaticSpa ? "COPY nginx.conf /etc/nginx/nginx.conf" : "",
`COPY ${publishDirectory || "."} .`,
'CMD ["nginx", "-g", "daemon off;"]',
].join("\n"),
);
createFile(
buildAppDirectory,
".dockerignore",
[".git", ".env", "Dockerfile", ".dockerignore"].join("\n"),
);
await buildCustomDocker(
{
...application,
buildType: "dockerfile",
dockerfile: "Dockerfile",
},
writeStream,
);
return true;
} catch (e) {
throw e;
}
};
export const getStaticCommand = (
application: ApplicationNested,
logPath: string,
) => {
const { publishDirectory } = application;
const buildAppDirectory = getBuildAppDirectory(application);
command += getCreateFileCommand(
buildAppDirectory,
".dockerignore",
[".git", ".env", "Dockerfile", ".dockerignore"].join("\n"),
);
let command = getCreateFileCommand(
command += getCreateFileCommand(
buildAppDirectory,
"Dockerfile",
[
"FROM nginx:alpine",
"WORKDIR /usr/share/nginx/html/",
isStaticSpa ? "COPY nginx.conf /etc/nginx/nginx.conf" : "",
`COPY ${publishDirectory || "."} .`,
'CMD ["nginx", "-g", "daemon off;"]',
].join("\n"),
);
command += getDockerCommand(
{
...application,
buildType: "dockerfile",
dockerfile: "Dockerfile",
},
logPath,
);
command += getDockerCommand({
...application,
buildType: "dockerfile",
dockerfile: "Dockerfile",
});
return command;
};

View File

@@ -1,11 +1,6 @@
import type { WriteStream } from "node:fs";
import type { ApplicationNested } from "../builders";
import { spawnAsync } from "../process/spawnAsync";
export const uploadImage = async (
application: ApplicationNested,
writeStream: WriteStream,
) => {
export const uploadImageRemoteCommand = (application: ApplicationNested) => {
const registry = application.registry;
if (!registry) {
@@ -19,85 +14,28 @@ export const uploadImage = async (
const finalURL = registryUrl;
// Build registry tag in correct format: registry.com/owner/image:tag
// For ghcr.io: ghcr.io/username/image:tag
// For docker.io: docker.io/username/image:tag
const registryTag = imagePrefix
? `${registryUrl ? `${registryUrl}/` : ""}${imagePrefix}/${imageName}`
: `${registryUrl ? `${registryUrl}/` : ""}${username}/${imageName}`;
try {
writeStream.write(
`📦 [Enabled Registry] Uploading image to ${registry.registryType} | ${imageName} | ${finalURL} | ${registryTag}\n`,
);
const loginCommand = spawnAsync(
"docker",
["login", finalURL, "-u", registry.username, "--password-stdin"],
(data) => {
if (writeStream.writable) {
writeStream.write(data);
}
},
);
loginCommand.child?.stdin?.write(registry.password);
loginCommand.child?.stdin?.end();
await loginCommand;
await spawnAsync("docker", ["tag", imageName, registryTag], (data) => {
if (writeStream.writable) {
writeStream.write(data);
}
});
await spawnAsync("docker", ["push", registryTag], (data) => {
if (writeStream.writable) {
writeStream.write(data);
}
});
} catch (error) {
console.log(error);
throw error;
}
};
export const uploadImageRemoteCommand = (
application: ApplicationNested,
logPath: string,
) => {
const registry = application.registry;
if (!registry) {
throw new Error("Registry not found");
}
const { registryUrl, imagePrefix, username } = registry;
const { appName } = application;
const imageName = `${appName}:latest`;
const finalURL = registryUrl;
// Build registry tag in correct format: registry.com/owner/image:tag
const registryTag = imagePrefix
? `${registryUrl}/${imagePrefix}/${imageName}`
: `${registryUrl}/${username}/${imageName}`;
try {
const command = `
echo "📦 [Enabled Registry] Uploading image to '${registry.registryType}' | '${registryTag}'" >> ${logPath};
echo "${registry.password}" | docker login ${finalURL} -u ${registry.username} --password-stdin >> ${logPath} 2>> ${logPath} || {
echo "❌ DockerHub Failed" >> ${logPath};
echo "📦 [Enabled Registry] Uploading image to '${registry.registryType}' | '${registryTag}'" ;
echo "${registry.password}" | docker login ${finalURL} -u ${registry.username} --password-stdin || {
echo "❌ DockerHub Failed" ;
exit 1;
}
echo "✅ Registry Login Success" >> ${logPath};
docker tag ${imageName} ${registryTag} >> ${logPath} 2>> ${logPath} || {
echo "❌ Error tagging image" >> ${logPath};
echo "✅ Registry Login Success" ;
docker tag ${imageName} ${registryTag} || {
echo "❌ Error tagging image" ;
exit 1;
}
echo "✅ Image Tagged" >> ${logPath};
docker push ${registryTag} 2>> ${logPath} || {
echo "❌ Error pushing image" >> ${logPath};
echo "✅ Image Tagged" ;
docker push ${registryTag} || {
echo "❌ Error pushing image" ;
exit 1;
}
echo "✅ Image Pushed" >> ${logPath};
echo "✅ Image Pushed" ;
`;
return command;
} catch (error) {

View File

@@ -46,6 +46,7 @@ export const buildMariadb = async (mariadb: MariadbNested) => {
UpdateConfig,
Networks,
StopGracePeriod,
EndpointSpec,
} = generateConfigContainer(mariadb);
const resources = calculateResources({
memoryLimit,
@@ -89,19 +90,21 @@ export const buildMariadb = async (mariadb: MariadbNested) => {
},
Mode,
RollbackConfig,
EndpointSpec: {
Mode: "dnsrr",
Ports: externalPort
? [
{
Protocol: "tcp",
TargetPort: 3306,
PublishedPort: externalPort,
PublishMode: "host",
},
]
: [],
},
EndpointSpec: EndpointSpec
? EndpointSpec
: {
Mode: "dnsrr" as const,
Ports: externalPort
? [
{
Protocol: "tcp" as const,
TargetPort: 3306,
PublishedPort: externalPort,
PublishMode: "host" as const,
},
]
: [],
},
UpdateConfig,
...(StopGracePeriod !== undefined &&
StopGracePeriod !== null && { StopGracePeriod }),

View File

@@ -92,6 +92,7 @@ ${command ?? "wait $MONGOD_PID"}`;
UpdateConfig,
Networks,
StopGracePeriod,
EndpointSpec,
} = generateConfigContainer(mongo);
const resources = calculateResources({
@@ -142,19 +143,21 @@ ${command ?? "wait $MONGOD_PID"}`;
},
Mode,
RollbackConfig,
EndpointSpec: {
Mode: "dnsrr",
Ports: externalPort
? [
{
Protocol: "tcp",
TargetPort: 27017,
PublishedPort: externalPort,
PublishMode: "host",
},
]
: [],
},
EndpointSpec: EndpointSpec
? EndpointSpec
: {
Mode: "dnsrr" as const,
Ports: externalPort
? [
{
Protocol: "tcp" as const,
TargetPort: 27017,
PublishedPort: externalPort,
PublishMode: "host" as const,
},
]
: [],
},
UpdateConfig,
...(StopGracePeriod !== undefined &&
StopGracePeriod !== null && { StopGracePeriod }),

View File

@@ -52,6 +52,7 @@ export const buildMysql = async (mysql: MysqlNested) => {
UpdateConfig,
Networks,
StopGracePeriod,
EndpointSpec,
} = generateConfigContainer(mysql);
const resources = calculateResources({
memoryLimit,
@@ -95,19 +96,21 @@ export const buildMysql = async (mysql: MysqlNested) => {
},
Mode,
RollbackConfig,
EndpointSpec: {
Mode: "dnsrr",
Ports: externalPort
? [
{
Protocol: "tcp",
TargetPort: 3306,
PublishedPort: externalPort,
PublishMode: "host",
},
]
: [],
},
EndpointSpec: EndpointSpec
? EndpointSpec
: {
Mode: "dnsrr" as const,
Ports: externalPort
? [
{
Protocol: "tcp" as const,
TargetPort: 3306,
PublishedPort: externalPort,
PublishMode: "host" as const,
},
]
: [],
},
UpdateConfig,
...(StopGracePeriod !== undefined &&
StopGracePeriod !== null && { StopGracePeriod }),

View File

@@ -45,6 +45,7 @@ export const buildPostgres = async (postgres: PostgresNested) => {
UpdateConfig,
Networks,
StopGracePeriod,
EndpointSpec,
} = generateConfigContainer(postgres);
const resources = calculateResources({
memoryLimit,
@@ -88,19 +89,21 @@ export const buildPostgres = async (postgres: PostgresNested) => {
},
Mode,
RollbackConfig,
EndpointSpec: {
Mode: "dnsrr",
Ports: externalPort
? [
{
Protocol: "tcp",
TargetPort: 5432,
PublishedPort: externalPort,
PublishMode: "host",
},
]
: [],
},
EndpointSpec: EndpointSpec
? EndpointSpec
: {
Mode: "dnsrr" as const,
Ports: externalPort
? [
{
Protocol: "tcp" as const,
TargetPort: 5432,
PublishedPort: externalPort,
PublishMode: "host" as const,
},
]
: [],
},
UpdateConfig,
...(StopGracePeriod !== undefined &&
StopGracePeriod !== null && { StopGracePeriod }),

View File

@@ -43,6 +43,7 @@ export const buildRedis = async (redis: RedisNested) => {
UpdateConfig,
Networks,
StopGracePeriod,
EndpointSpec,
} = generateConfigContainer(redis);
const resources = calculateResources({
memoryLimit,
@@ -85,19 +86,21 @@ export const buildRedis = async (redis: RedisNested) => {
},
Mode,
RollbackConfig,
EndpointSpec: {
Mode: "dnsrr",
Ports: externalPort
? [
{
Protocol: "tcp",
TargetPort: 6379,
PublishedPort: externalPort,
PublishMode: "host",
},
]
: [],
},
EndpointSpec: EndpointSpec
? EndpointSpec
: {
Mode: "dnsrr" as const,
Ports: externalPort
? [
{
Protocol: "tcp" as const,
TargetPort: 6379,
PublishedPort: externalPort,
PublishMode: "host" as const,
},
]
: [],
},
UpdateConfig,
...(StopGracePeriod !== undefined &&
StopGracePeriod !== null && { StopGracePeriod }),

View File

@@ -1,11 +1,11 @@
import { findComposeById } from "@dokploy/server/services/compose";
import { stringify } from "yaml";
import { execAsync, execAsyncRemote } from "../process/execAsync";
import { addAppNameToAllServiceNames } from "./collision/root-network";
import { generateRandomHash } from "./compose";
import { addSuffixToAllVolumes } from "./compose/volume";
import {
cloneCompose,
cloneComposeRemote,
loadDockerCompose,
loadDockerComposeRemote,
} from "./domain";
@@ -31,10 +31,11 @@ export const randomizeIsolatedDeploymentComposeFile = async (
) => {
const compose = await findComposeById(composeId);
const command = await cloneCompose(compose);
if (compose.serverId) {
await cloneComposeRemote(compose);
await execAsyncRemote(compose.serverId, command);
} else {
await cloneCompose(compose);
await execAsync(command);
}
let composeData: ComposeSpecification | null;

View File

@@ -1,35 +1,16 @@
import fs, { existsSync, readFileSync } from "node:fs";
import { writeFile } from "node:fs/promises";
import { join } from "node:path";
import { paths } from "@dokploy/server/constants";
import type { Compose } from "@dokploy/server/services/compose";
import type { Domain } from "@dokploy/server/services/domain";
import { parse, stringify } from "yaml";
import { execAsyncRemote } from "../process/execAsync";
import {
cloneRawBitbucketRepository,
cloneRawBitbucketRepositoryRemote,
} from "../providers/bitbucket";
import {
cloneGitRawRepository,
cloneRawGitRepositoryRemote,
} from "../providers/git";
import {
cloneRawGiteaRepository,
cloneRawGiteaRepositoryRemote,
} from "../providers/gitea";
import {
cloneRawGithubRepository,
cloneRawGithubRepositoryRemote,
} from "../providers/github";
import {
cloneRawGitlabRepository,
cloneRawGitlabRepositoryRemote,
} from "../providers/gitlab";
import {
createComposeFileRaw,
createComposeFileRawRemote,
} from "../providers/raw";
import { cloneBitbucketRepository } from "../providers/bitbucket";
import { cloneGitRepository } from "../providers/git";
import { cloneGiteaRepository } from "../providers/gitea";
import { cloneGithubRepository } from "../providers/github";
import { cloneGitlabRepository } from "../providers/gitlab";
import { getCreateComposeFileCommand } from "../providers/raw";
import { randomizeDeployableSpecificationFile } from "./collision";
import { randomizeSpecificationFile } from "./compose";
import type {
@@ -40,35 +21,25 @@ import type {
import { encodeBase64 } from "./utils";
export const cloneCompose = async (compose: Compose) => {
let command = "set -e;";
const entity = {
...compose,
type: "compose" as const,
};
if (compose.sourceType === "github") {
await cloneRawGithubRepository(compose);
command += await cloneGithubRepository(entity);
} else if (compose.sourceType === "gitlab") {
await cloneRawGitlabRepository(compose);
command += await cloneGitlabRepository(entity);
} else if (compose.sourceType === "bitbucket") {
await cloneRawBitbucketRepository(compose);
command += await cloneBitbucketRepository(entity);
} else if (compose.sourceType === "git") {
await cloneGitRawRepository(compose);
command += await cloneGitRepository(entity);
} else if (compose.sourceType === "gitea") {
await cloneRawGiteaRepository(compose);
command += await cloneGiteaRepository(entity);
} else if (compose.sourceType === "raw") {
await createComposeFileRaw(compose);
}
};
export const cloneComposeRemote = async (compose: Compose) => {
if (compose.sourceType === "github") {
await cloneRawGithubRepositoryRemote(compose);
} else if (compose.sourceType === "gitlab") {
await cloneRawGitlabRepositoryRemote(compose);
} else if (compose.sourceType === "bitbucket") {
await cloneRawBitbucketRepositoryRemote(compose);
} else if (compose.sourceType === "git") {
await cloneRawGitRepositoryRemote(compose);
} else if (compose.sourceType === "gitea") {
await cloneRawGiteaRepositoryRemote(compose);
} else if (compose.sourceType === "raw") {
await createComposeFileRawRemote(compose);
command += getCreateComposeFileCommand(compose);
}
return command;
};
export const getComposePath = (compose: Compose) => {
@@ -131,28 +102,9 @@ export const readComposeFile = async (compose: Compose) => {
return null;
};
export const writeDomainsToCompose = async (
compose: Compose,
domains: Domain[],
) => {
if (!domains.length) {
return;
}
const composeConverted = await addDomainToCompose(compose, domains);
const path = getComposePath(compose);
const composeString = stringify(composeConverted, { lineWidth: 1000 });
try {
await writeFile(path, composeString, "utf8");
} catch (error) {
throw error;
}
};
export const writeDomainsToComposeRemote = async (
compose: Compose,
domains: Domain[],
logPath: string,
) => {
if (!domains.length) {
return "";
@@ -164,7 +116,7 @@ export const writeDomainsToComposeRemote = async (
if (!composeConverted) {
return `
echo "❌ Error: Compose file not found" >> ${logPath};
echo "❌ Error: Compose file not found";
exit 1;
`;
}
@@ -175,12 +127,13 @@ exit 1;
}
} catch (error) {
// @ts-ignore
return `echo "❌ Has occured an error: ${error?.message || error}" >> ${logPath};
return `echo "❌ Has occured an error: ${error?.message || error}";
exit 1;
`;
}
return "";
};
// (node:59875) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 SIGTERM listeners added to [process]. Use emitter.setMaxListeners() to increase limit
export const addDomainToCompose = async (
compose: Compose,
domains: Domain[],
@@ -190,7 +143,7 @@ export const addDomainToCompose = async (
let result: ComposeSpecification | null;
if (compose.serverId) {
result = await loadDockerComposeRemote(compose); // aca hay que ir al servidor e ir a traer el compose file al servidor
result = await loadDockerComposeRemote(compose);
} else {
result = await loadDockerCompose(compose);
}

View File

@@ -395,6 +395,7 @@ export const generateConfigContainer = (
mounts,
networkSwarm,
stopGracePeriodSwarm,
endpointSpecSwarm,
} = application;
const sanitizedStopGracePeriodSwarm =
@@ -408,11 +409,9 @@ export const generateConfigContainer = (
...(healthCheckSwarm && {
HealthCheck: healthCheckSwarm,
}),
...(restartPolicySwarm
? {
RestartPolicy: restartPolicySwarm,
}
: {}),
...(restartPolicySwarm && {
RestartPolicy: restartPolicySwarm,
}),
...(placementSwarm
? {
Placement: placementSwarm,
@@ -461,6 +460,18 @@ export const generateConfigContainer = (
: {
Networks: [{ Target: "dokploy-network" }],
}),
...(endpointSpecSwarm && {
EndpointSpec: {
...(endpointSpecSwarm.Mode && { Mode: endpointSpecSwarm.Mode }),
Ports:
endpointSpecSwarm.Ports?.map((port) => ({
Protocol: (port.Protocol || "tcp") as "tcp" | "udp" | "sctp",
TargetPort: port.TargetPort || 0,
PublishedPort: port.PublishedPort || 0,
PublishMode: (port.PublishMode || "host") as "ingress" | "host",
})) || [],
},
}),
};
};

View File

@@ -116,11 +116,7 @@ export const execAsyncRemote = async (
if (code === 0) {
resolve({ stdout, stderr });
} else {
reject(
new Error(
`Command exited with code ${code}. Stderr: ${stderr}, command: ${command}`,
),
);
reject(new Error(`Error occurred ❌: ${stderr}`));
}
})
.on("data", (data: string) => {

View File

@@ -1,4 +1,3 @@
import { createWriteStream } from "node:fs";
import { join } from "node:path";
import { paths } from "@dokploy/server/constants";
import type {
@@ -9,12 +8,8 @@ import {
type Bitbucket,
findBitbucketById,
} from "@dokploy/server/services/bitbucket";
import type { Compose } from "@dokploy/server/services/compose";
import type { InferResultType } from "@dokploy/server/types/with";
import { TRPCError } from "@trpc/server";
import { recreateDirectory } from "../filesystem/directory";
import { execAsyncRemote } from "../process/execAsync";
import { spawnAsync } from "../process/spawnAsync";
export type ApplicationWithBitbucket = InferResultType<
"applications",
@@ -81,202 +76,52 @@ export const getBitbucketHeaders = (bitbucketProvider: Bitbucket) => {
};
};
export const cloneBitbucketRepository = async (
entity: ApplicationWithBitbucket | ComposeWithBitbucket,
logPath: string,
isCompose = false,
) => {
const { COMPOSE_PATH, APPLICATIONS_PATH } = paths();
const writeStream = createWriteStream(logPath, { flags: "a" });
interface CloneBitbucketRepository {
appName: string;
bitbucketRepository: string | null;
bitbucketOwner: string | null;
bitbucketBranch: string | null;
bitbucketId: string | null;
enableSubmodules: boolean;
serverId: string | null;
type?: "application" | "compose";
}
export const cloneBitbucketRepository = async ({
type = "application",
...entity
}: CloneBitbucketRepository) => {
let command = "set -e;";
const {
appName,
bitbucketRepository,
bitbucketOwner,
bitbucketBranch,
bitbucketId,
bitbucket,
enableSubmodules,
serverId,
} = entity;
const { COMPOSE_PATH, APPLICATIONS_PATH } = paths(!!serverId);
if (!bitbucketId) {
throw new TRPCError({
code: "NOT_FOUND",
message: "Bitbucket Provider not found",
});
command += `echo "Error: ❌ Bitbucket Provider not found"; exit 1;`;
return command;
}
const bitbucket = await findBitbucketById(bitbucketId);
const basePath = isCompose ? COMPOSE_PATH : APPLICATIONS_PATH;
if (!bitbucket) {
command += `echo "Error: ❌ Bitbucket Provider not found"; exit 1;`;
return command;
}
const basePath = type === "compose" ? COMPOSE_PATH : APPLICATIONS_PATH;
const outputPath = join(basePath, appName, "code");
await recreateDirectory(outputPath);
command += `rm -rf ${outputPath};`;
command += `mkdir -p ${outputPath};`;
const repoclone = `bitbucket.org/${bitbucketOwner}/${bitbucketRepository}.git`;
const cloneUrl = getBitbucketCloneUrl(bitbucket, repoclone);
try {
writeStream.write(`\nCloning Repo ${repoclone} to ${outputPath}: ✅\n`);
const cloneArgs = [
"clone",
"--branch",
bitbucketBranch!,
"--depth",
"1",
...(enableSubmodules ? ["--recurse-submodules"] : []),
cloneUrl,
outputPath,
"--progress",
];
await spawnAsync("git", cloneArgs, (data) => {
if (writeStream.writable) {
writeStream.write(data);
}
});
writeStream.write(`\nCloned ${repoclone} to ${outputPath}: ✅\n`);
} catch (error) {
writeStream.write(`ERROR Cloning: ${error}: ❌`);
throw error;
} finally {
writeStream.end();
}
};
export const cloneRawBitbucketRepository = async (entity: Compose) => {
const { COMPOSE_PATH } = paths();
const {
appName,
bitbucketRepository,
bitbucketOwner,
bitbucketBranch,
bitbucketId,
enableSubmodules,
} = entity;
if (!bitbucketId) {
throw new TRPCError({
code: "NOT_FOUND",
message: "Bitbucket Provider not found",
});
}
const bitbucketProvider = await findBitbucketById(bitbucketId);
const basePath = COMPOSE_PATH;
const outputPath = join(basePath, appName, "code");
await recreateDirectory(outputPath);
const repoclone = `bitbucket.org/${bitbucketOwner}/${bitbucketRepository}.git`;
const cloneUrl = getBitbucketCloneUrl(bitbucketProvider, repoclone);
try {
const cloneArgs = [
"clone",
"--branch",
bitbucketBranch!,
"--depth",
"1",
...(enableSubmodules ? ["--recurse-submodules"] : []),
cloneUrl,
outputPath,
"--progress",
];
await spawnAsync("git", cloneArgs);
} catch (error) {
throw error;
}
};
export const cloneRawBitbucketRepositoryRemote = async (compose: Compose) => {
const { COMPOSE_PATH } = paths(true);
const {
appName,
bitbucketRepository,
bitbucketOwner,
bitbucketBranch,
bitbucketId,
serverId,
enableSubmodules,
} = compose;
if (!serverId) {
throw new TRPCError({
code: "NOT_FOUND",
message: "Server not found",
});
}
if (!bitbucketId) {
throw new TRPCError({
code: "NOT_FOUND",
message: "Bitbucket Provider not found",
});
}
const bitbucketProvider = await findBitbucketById(bitbucketId);
const basePath = COMPOSE_PATH;
const outputPath = join(basePath, appName, "code");
const repoclone = `bitbucket.org/${bitbucketOwner}/${bitbucketRepository}.git`;
const cloneUrl = getBitbucketCloneUrl(bitbucketProvider, repoclone);
try {
const cloneCommand = `
rm -rf ${outputPath};
git clone --branch ${bitbucketBranch} --depth 1 ${enableSubmodules ? "--recurse-submodules" : ""} ${cloneUrl} ${outputPath}
`;
await execAsyncRemote(serverId, cloneCommand);
} catch (error) {
throw error;
}
};
export const getBitbucketCloneCommand = async (
entity: ApplicationWithBitbucket | ComposeWithBitbucket,
logPath: string,
isCompose = false,
) => {
const { COMPOSE_PATH, APPLICATIONS_PATH } = paths(true);
const {
appName,
bitbucketRepository,
bitbucketOwner,
bitbucketBranch,
bitbucketId,
serverId,
enableSubmodules,
} = entity;
if (!serverId) {
throw new TRPCError({
code: "NOT_FOUND",
message: "Server not found",
});
}
if (!bitbucketId) {
const command = `
echo "Error: ❌ Bitbucket Provider not found" >> ${logPath};
exit 1;
`;
await execAsyncRemote(serverId, command);
throw new TRPCError({
code: "NOT_FOUND",
message: "Bitbucket Provider not found",
});
}
const bitbucketProvider = await findBitbucketById(bitbucketId);
const basePath = isCompose ? COMPOSE_PATH : APPLICATIONS_PATH;
const outputPath = join(basePath, appName, "code");
await recreateDirectory(outputPath);
const repoclone = `bitbucket.org/${bitbucketOwner}/${bitbucketRepository}.git`;
const cloneUrl = getBitbucketCloneUrl(bitbucketProvider, repoclone);
const cloneCommand = `
rm -rf ${outputPath};
mkdir -p ${outputPath};
if ! git clone --branch ${bitbucketBranch} --depth 1 ${enableSubmodules ? "--recurse-submodules" : ""} --progress ${cloneUrl} ${outputPath} >> ${logPath} 2>&1; then
echo "❌ [ERROR] Fail to clone the repository ${repoclone}" >> ${logPath};
exit 1;
fi
echo "Cloned ${repoclone} to ${outputPath}: ✅" >> ${logPath};
`;
return cloneCommand;
command += `echo "Cloning Repo ${repoclone} to ${outputPath}: ✅";`;
command += `git clone --branch ${bitbucketBranch} --depth 1 ${enableSubmodules ? "--recurse-submodules" : ""} ${cloneUrl} ${outputPath} --progress;`;
return command;
};
export const getBitbucketRepositories = async (bitbucketId?: string) => {

View File

@@ -1,60 +1,6 @@
import { createWriteStream } from "node:fs";
import { type ApplicationNested, mechanizeDockerContainer } from "../builders";
import { pullImage } from "../docker/utils";
import type { ApplicationNested } from "../builders";
interface RegistryAuth {
username: string;
password: string;
registryUrl: string;
}
export const buildDocker = async (
application: ApplicationNested,
logPath: string,
): Promise<void> => {
const { buildType, dockerImage, username, password } = application;
const authConfig: Partial<RegistryAuth> = {
username: username || "",
password: password || "",
registryUrl: application.registryUrl || "",
};
const writeStream = createWriteStream(logPath, { flags: "a" });
writeStream.write(`\nBuild ${buildType}\n`);
writeStream.write(`Pulling ${dockerImage}: ✅\n`);
try {
if (!dockerImage) {
throw new Error("Docker image not found");
}
await pullImage(
dockerImage,
(data) => {
if (writeStream.writable) {
writeStream.write(`${data}\n`);
}
},
authConfig,
);
await mechanizeDockerContainer(application);
writeStream.write("\nDocker Deployed: ✅\n");
} catch (error) {
writeStream.write(
`❌ Error: ${error instanceof Error ? error.message : String(error)}`,
);
throw error;
} finally {
writeStream.end();
}
};
export const buildRemoteDocker = async (
application: ApplicationNested,
logPath: string,
) => {
export const buildRemoteDocker = async (application: ApplicationNested) => {
const { registryUrl, dockerImage, username, password } = application;
try {
@@ -62,25 +8,25 @@ export const buildRemoteDocker = async (
throw new Error("Docker image not found");
}
let command = `
echo "Pulling ${dockerImage}" >> ${logPath};
echo "Pulling ${dockerImage}";
`;
if (username && password) {
command += `
if ! echo "${password}" | docker login --username "${username}" --password-stdin "${registryUrl || ""}" >> ${logPath} 2>&1; then
echo "❌ Login failed" >> ${logPath};
if ! echo "${password}" | docker login --username "${username}" --password-stdin "${registryUrl || ""}" 2>&1; then
echo "❌ Login failed";
exit 1;
fi
`;
}
command += `
docker pull ${dockerImage} >> ${logPath} 2>> ${logPath} || {
echo "❌ Pulling image failed" >> ${logPath};
docker pull ${dockerImage} 2>&1 || {
echo "❌ Pulling image failed";
exit 1;
}
echo "✅ Pulling image completed." >> ${logPath};
echo "✅ Pulling image completed.";
`;
return command;
} catch (error) {

View File

@@ -1,159 +1,65 @@
import { createWriteStream } from "node:fs";
import path, { join } from "node:path";
import { paths } from "@dokploy/server/constants";
import type { Compose } from "@dokploy/server/services/compose";
import {
findSSHKeyById,
updateSSHKeyById,
} from "@dokploy/server/services/ssh-key";
import { TRPCError } from "@trpc/server";
import { recreateDirectory } from "../filesystem/directory";
import { execAsync, execAsyncRemote } from "../process/execAsync";
import { spawnAsync } from "../process/spawnAsync";
export const cloneGitRepository = async (
entity: {
appName: string;
customGitUrl?: string | null;
customGitBranch?: string | null;
customGitSSHKeyId?: string | null;
enableSubmodules?: boolean;
},
logPath: string,
isCompose = false,
) => {
const { SSH_PATH, COMPOSE_PATH, APPLICATIONS_PATH } = paths();
interface CloneGitRepository {
appName: string;
customGitUrl?: string | null;
customGitBranch?: string | null;
customGitSSHKeyId?: string | null;
enableSubmodules?: boolean;
serverId: string | null;
type?: "application" | "compose";
}
export const cloneGitRepository = async ({
type = "application",
...entity
}: CloneGitRepository) => {
let command = "set -e;";
const {
appName,
customGitUrl,
customGitBranch,
customGitSSHKeyId,
enableSubmodules,
serverId,
} = entity;
const { SSH_PATH, COMPOSE_PATH, APPLICATIONS_PATH } = paths(!!serverId);
if (!customGitUrl || !customGitBranch) {
throw new TRPCError({
code: "BAD_REQUEST",
message: "Error: Repository not found",
});
command += `echo "Error: ❌ Repository not found"; exit 1;`;
return command;
}
const writeStream = createWriteStream(logPath, { flags: "a" });
const temporalKeyPath = path.join("/tmp", "id_rsa");
if (customGitSSHKeyId) {
const sshKey = await findSSHKeyById(customGitSSHKeyId);
await execAsync(`
command += `
echo "${sshKey.privateKey}" > ${temporalKeyPath}
chmod 600 ${temporalKeyPath}
`);
chmod 600 ${temporalKeyPath};
`;
}
const basePath = isCompose ? COMPOSE_PATH : APPLICATIONS_PATH;
const basePath = type === "compose" ? COMPOSE_PATH : APPLICATIONS_PATH;
const outputPath = join(basePath, appName, "code");
const knownHostsPath = path.join(SSH_PATH, "known_hosts");
try {
if (!isHttpOrHttps(customGitUrl)) {
if (!customGitSSHKeyId) {
throw new TRPCError({
code: "BAD_REQUEST",
message:
"Error: you are trying to clone a ssh repository without a ssh key, please set a ssh key",
});
}
await addHostToKnownHosts(customGitUrl);
if (!isHttpOrHttps(customGitUrl)) {
if (!customGitSSHKeyId) {
command += `echo "Error: ❌ You are trying to clone a ssh repository without a ssh key, please set a ssh key"; exit 1;`;
return command;
}
await recreateDirectory(outputPath);
writeStream.write(
`\nCloning Repo Custom ${customGitUrl} to ${outputPath}: ✅\n`,
);
if (customGitSSHKeyId) {
await updateSSHKeyById({
sshKeyId: customGitSSHKeyId,
lastUsedAt: new Date().toISOString(),
});
}
const { port } = sanitizeRepoPathSSH(customGitUrl);
const cloneArgs = [
"clone",
"--branch",
customGitBranch,
"--depth",
"1",
...(enableSubmodules ? ["--recurse-submodules"] : []),
customGitUrl,
outputPath,
"--progress",
];
await spawnAsync(
"git",
cloneArgs,
(data) => {
if (writeStream.writable) {
writeStream.write(data);
}
},
{
env: {
...process.env,
...(customGitSSHKeyId && {
GIT_SSH_COMMAND: `ssh -i ${temporalKeyPath}${port ? ` -p ${port}` : ""} -o UserKnownHostsFile=${knownHostsPath}`,
}),
},
},
);
writeStream.write(`\nCloned Custom Git ${customGitUrl}: ✅\n`);
} catch (error) {
writeStream.write(`\nERROR Cloning Custom Git: ${error}: ❌\n`);
throw error;
} finally {
writeStream.end();
command += addHostToKnownHostsCommand(customGitUrl);
}
};
export const getCustomGitCloneCommand = async (
entity: {
appName: string;
customGitUrl?: string | null;
customGitBranch?: string | null;
customGitSSHKeyId?: string | null;
serverId: string | null;
enableSubmodules: boolean;
},
logPath: string,
isCompose = false,
) => {
const { SSH_PATH, COMPOSE_PATH, APPLICATIONS_PATH } = paths(true);
const {
appName,
customGitUrl,
customGitBranch,
customGitSSHKeyId,
serverId,
enableSubmodules,
} = entity;
if (!customGitUrl || !customGitBranch) {
const command = `
echo "Error: ❌ Repository not found" >> ${logPath};
exit 1;
`;
await execAsyncRemote(serverId, command);
throw new TRPCError({
code: "BAD_REQUEST",
message: "Error: Repository not found",
});
}
const basePath = isCompose ? COMPOSE_PATH : APPLICATIONS_PATH;
const outputPath = join(basePath, appName, "code");
const knownHostsPath = path.join(SSH_PATH, "known_hosts");
command += `rm -rf ${outputPath};`;
command += `mkdir -p ${outputPath};`;
command += `echo "Cloning Repo Custom ${customGitUrl} to ${outputPath}: ✅";`;
if (customGitSSHKeyId) {
await updateSSHKeyById({
@@ -161,48 +67,22 @@ export const getCustomGitCloneCommand = async (
lastUsedAt: new Date().toISOString(),
});
}
try {
const command = [];
if (!isHttpOrHttps(customGitUrl)) {
if (!customGitSSHKeyId) {
command.push(
`echo "Error: you are trying to clone a ssh repository without a ssh key, please set a ssh key ❌" >> ${logPath};
exit 1;
`,
);
}
command.push(addHostToKnownHostsCommand(customGitUrl));
}
command.push(`rm -rf ${outputPath};`);
command.push(`mkdir -p ${outputPath};`);
command.push(
`echo "Cloning Custom Git ${customGitUrl}" to ${outputPath}: ✅ >> ${logPath};`,
);
if (customGitSSHKeyId) {
const sshKey = await findSSHKeyById(customGitSSHKeyId);
const { port } = sanitizeRepoPathSSH(customGitUrl);
const gitSshCommand = `ssh -i /tmp/id_rsa${port ? ` -p ${port}` : ""} -o UserKnownHostsFile=${knownHostsPath}`;
command.push(
`
echo "${sshKey.privateKey}" > /tmp/id_rsa
chmod 600 /tmp/id_rsa
export GIT_SSH_COMMAND="${gitSshCommand}"
`,
);
}
command.push(
`if ! git clone --branch ${customGitBranch} --depth 1 ${enableSubmodules ? "--recurse-submodules" : ""} --progress ${customGitUrl} ${outputPath} >> ${logPath} 2>&1; then
echo "❌ [ERROR] Fail to clone the repository ${customGitUrl}" >> ${logPath};
if (customGitSSHKeyId) {
const sshKey = await findSSHKeyById(customGitSSHKeyId);
const { port } = sanitizeRepoPathSSH(customGitUrl);
const gitSshCommand = `ssh -i /tmp/id_rsa${port ? ` -p ${port}` : ""} -o UserKnownHostsFile=${knownHostsPath}`;
command += `echo "${sshKey.privateKey}" > /tmp/id_rsa;`;
command += "chmod 600 /tmp/id_rsa;";
command += `export GIT_SSH_COMMAND="${gitSshCommand}";`;
}
command += `if ! git clone --branch ${customGitBranch} --depth 1 ${enableSubmodules ? "--recurse-submodules" : ""} --progress ${customGitUrl} ${outputPath}; then
echo "❌ [ERROR] Fail to clone the repository ${customGitUrl}";
exit 1;
fi
`,
);
command.push(`echo "Cloned Custom Git ${customGitUrl}: ✅" >> ${logPath};`);
return command.join("\n");
} catch (error) {
throw error;
}
`;
return command;
};
const isHttpOrHttps = (url: string): boolean => {
@@ -210,19 +90,19 @@ const isHttpOrHttps = (url: string): boolean => {
return regex.test(url);
};
const addHostToKnownHosts = async (repositoryURL: string) => {
const { SSH_PATH } = paths();
const { domain, port } = sanitizeRepoPathSSH(repositoryURL);
const knownHostsPath = path.join(SSH_PATH, "known_hosts");
// const addHostToKnownHosts = async (repositoryURL: string) => {
// const { SSH_PATH } = paths();
// const { domain, port } = sanitizeRepoPathSSH(repositoryURL);
// const knownHostsPath = path.join(SSH_PATH, "known_hosts");
const command = `ssh-keyscan -p ${port} ${domain} >> ${knownHostsPath}`;
try {
await execAsync(command);
} catch (error) {
console.error(`Error adding host to known_hosts: ${error}`);
throw error;
}
};
// const command = `ssh-keyscan -p ${port} ${domain} >> ${knownHostsPath}`;
// try {
// await execAsync(command);
// } catch (error) {
// console.error(`Error adding host to known_hosts: ${error}`);
// throw error;
// }
// };
const addHostToKnownHostsCommand = (repositoryURL: string) => {
const { SSH_PATH } = paths(true);
@@ -267,160 +147,43 @@ const sanitizeRepoPathSSH = (input: string) => {
};
};
export const cloneGitRawRepository = async (entity: {
interface Props {
appName: string;
customGitUrl?: string | null;
customGitBranch?: string | null;
customGitSSHKeyId?: string | null;
enableSubmodules?: boolean;
}) => {
const {
appName,
customGitUrl,
customGitBranch,
customGitSSHKeyId,
enableSubmodules,
} = entity;
type?: "application" | "compose";
serverId: string | null;
}
if (!customGitUrl || !customGitBranch) {
throw new TRPCError({
code: "BAD_REQUEST",
message: "Error: Repository not found",
});
}
const { SSH_PATH, COMPOSE_PATH } = paths();
const temporalKeyPath = path.join("/tmp", "id_rsa");
const basePath = COMPOSE_PATH;
export const getGitCommitInfo = async ({
appName,
type = "application",
serverId,
}: Props) => {
const { COMPOSE_PATH, APPLICATIONS_PATH } = paths(!!serverId);
const basePath = type === "compose" ? COMPOSE_PATH : APPLICATIONS_PATH;
const outputPath = join(basePath, appName, "code");
const knownHostsPath = path.join(SSH_PATH, "known_hosts");
if (customGitSSHKeyId) {
const sshKey = await findSSHKeyById(customGitSSHKeyId);
await execAsync(`
echo "${sshKey.privateKey}" > ${temporalKeyPath}
chmod 600 ${temporalKeyPath}
`);
}
let stdoutResult = "";
const result = {
message: "",
hash: "",
};
try {
if (!isHttpOrHttps(customGitUrl)) {
if (!customGitSSHKeyId) {
throw new TRPCError({
code: "BAD_REQUEST",
message:
"Error: you are trying to clone a ssh repository without a ssh key, please set a ssh key",
});
}
await addHostToKnownHosts(customGitUrl);
}
await recreateDirectory(outputPath);
if (customGitSSHKeyId) {
await updateSSHKeyById({
sshKeyId: customGitSSHKeyId,
lastUsedAt: new Date().toISOString(),
});
const gitCommand = `git -C ${outputPath} log -1 --pretty=format:"%H---DELIMITER---%B"`;
if (serverId) {
const { stdout } = await execAsyncRemote(serverId, gitCommand);
stdoutResult = stdout.trim();
} else {
const { stdout } = await execAsync(gitCommand);
stdoutResult = stdout.trim();
}
const { port } = sanitizeRepoPathSSH(customGitUrl);
const cloneArgs = [
"clone",
"--branch",
customGitBranch,
"--depth",
"1",
...(enableSubmodules ? ["--recurse-submodules"] : []),
customGitUrl,
outputPath,
"--progress",
];
await spawnAsync("git", cloneArgs, (_data) => {}, {
env: {
...process.env,
...(customGitSSHKeyId && {
GIT_SSH_COMMAND: `ssh -i ${temporalKeyPath}${port ? ` -p ${port}` : ""} -o UserKnownHostsFile=${knownHostsPath}`,
}),
},
});
const parts = stdoutResult.split("---DELIMITER---");
if (parts && parts.length === 2) {
result.hash = parts[0]?.trim() || "";
result.message = parts[1]?.trim() || "";
}
} catch (error) {
throw error;
}
};
export const cloneRawGitRepositoryRemote = async (compose: Compose) => {
const {
appName,
customGitBranch,
customGitUrl,
customGitSSHKeyId,
serverId,
enableSubmodules,
} = compose;
if (!serverId) {
throw new TRPCError({
code: "NOT_FOUND",
message: "Server not found",
});
}
if (!customGitUrl) {
throw new TRPCError({
code: "NOT_FOUND",
message: "Git Provider not found",
});
}
const { SSH_PATH, COMPOSE_PATH } = paths(true);
const basePath = COMPOSE_PATH;
const outputPath = join(basePath, appName, "code");
const knownHostsPath = path.join(SSH_PATH, "known_hosts");
if (customGitSSHKeyId) {
await updateSSHKeyById({
sshKeyId: customGitSSHKeyId,
lastUsedAt: new Date().toISOString(),
});
}
try {
const command = [];
if (!isHttpOrHttps(customGitUrl)) {
if (!customGitSSHKeyId) {
command.push(
`echo "Error: you are trying to clone a ssh repository without a ssh key, please set a ssh key ❌" ;
exit 1;
`,
);
}
command.push(addHostToKnownHostsCommand(customGitUrl));
}
command.push(`rm -rf ${outputPath};`);
command.push(`mkdir -p ${outputPath};`);
if (customGitSSHKeyId) {
const sshKey = await findSSHKeyById(customGitSSHKeyId);
const { port } = sanitizeRepoPathSSH(customGitUrl);
const gitSshCommand = `ssh -i /tmp/id_rsa${port ? ` -p ${port}` : ""} -o UserKnownHostsFile=${knownHostsPath}`;
command.push(
`
echo "${sshKey.privateKey}" > /tmp/id_rsa
chmod 600 /tmp/id_rsa
export GIT_SSH_COMMAND="${gitSshCommand}"
`,
);
}
command.push(
`if ! git clone --branch ${customGitBranch} --depth 1 ${enableSubmodules ? "--recurse-submodules" : ""} --progress ${customGitUrl} ${outputPath} ; then
echo "[ERROR] Fail to clone the repository ";
exit 1;
fi
`,
);
await execAsyncRemote(serverId, command.join("\n"));
} catch (error) {
throw error;
console.error(`Error getting git commit info: ${error}`);
return null;
}
return result;
};

View File

@@ -1,7 +1,5 @@
import { createWriteStream } from "node:fs";
import { join } from "node:path";
import { paths } from "@dokploy/server/constants";
import type { Compose } from "@dokploy/server/services/compose";
import {
findGiteaById,
type Gitea,
@@ -9,9 +7,6 @@ import {
} from "@dokploy/server/services/gitea";
import type { InferResultType } from "@dokploy/server/types/with";
import { TRPCError } from "@trpc/server";
import { recreateDirectory } from "../filesystem/directory";
import { execAsyncRemote } from "../process/execAsync";
import { spawnAsync } from "../process/spawnAsync";
export const getErrorCloneRequirements = (entity: {
giteaRepository?: string | null;
@@ -119,79 +114,27 @@ export type ApplicationWithGitea = InferResultType<
export type ComposeWithGitea = InferResultType<"compose", { gitea: true }>;
export const getGiteaCloneCommand = async (
entity: ApplicationWithGitea | ComposeWithGitea,
logPath: string,
isCompose = false,
) => {
const {
appName,
giteaBranch,
giteaId,
giteaOwner,
giteaRepository,
serverId,
enableSubmodules,
} = entity;
if (!serverId) {
throw new TRPCError({
code: "NOT_FOUND",
message: "Server not found",
});
}
if (!giteaId) {
const command = `
echo "Error: ❌ Gitlab Provider not found" >> ${logPath};
exit 1;
`;
await execAsyncRemote(serverId, command);
throw new TRPCError({
code: "NOT_FOUND",
message: "Gitea Provider not found",
});
}
// Use paths(true) for remote operations
const { COMPOSE_PATH, APPLICATIONS_PATH } = paths(true);
await refreshGiteaToken(giteaId);
const gitea = await findGiteaById(giteaId);
const basePath = isCompose ? COMPOSE_PATH : APPLICATIONS_PATH;
const outputPath = join(basePath, appName, "code");
const repoClone = `${giteaOwner}/${giteaRepository}.git`;
const cloneUrl = buildGiteaCloneUrl(
gitea?.giteaUrl!,
gitea?.accessToken!,
giteaOwner!,
giteaRepository!,
);
const cloneCommand = `
rm -rf ${outputPath};
mkdir -p ${outputPath};
if ! git clone --branch ${giteaBranch} --depth 1 ${enableSubmodules ? "--recurse-submodules" : ""} ${cloneUrl} ${outputPath} >> ${logPath} 2>&1; then
echo "❌ [ERROR] Failed to clone the repository ${repoClone}" >> ${logPath};
exit 1;
fi
echo "Cloned ${repoClone} to ${outputPath}: ✅" >> ${logPath};
`;
return cloneCommand;
type GiteaClone = (ApplicationWithGitea | ComposeWithGitea) & {
serverId: string | null;
type?: "application" | "compose";
};
export const cloneGiteaRepository = async (
entity: ApplicationWithGitea | ComposeWithGitea,
logPath: string,
isCompose = false,
) => {
const { APPLICATIONS_PATH, COMPOSE_PATH } = paths();
interface CloneGiteaRepository {
appName: string;
giteaBranch: string | null;
giteaId: string | null;
giteaOwner: string | null;
giteaRepository: string | null;
enableSubmodules: boolean;
serverId: string | null;
type?: "application" | "compose";
}
const writeStream = createWriteStream(logPath, { flags: "a" });
export const cloneGiteaRepository = async ({
type = "application",
...entity
}: CloneGiteaRepository) => {
let command = "set -e;";
const {
appName,
giteaBranch,
@@ -199,27 +142,27 @@ export const cloneGiteaRepository = async (
giteaOwner,
giteaRepository,
enableSubmodules,
serverId,
} = entity;
const { APPLICATIONS_PATH, COMPOSE_PATH } = paths(!!serverId);
if (!giteaId) {
throw new TRPCError({
code: "NOT_FOUND",
message: "Gitea Provider not found",
});
command += `echo "Error: ❌ Gitea Provider not found"; exit 1;`;
return command;
}
await refreshGiteaToken(giteaId);
const giteaProvider = await findGiteaById(giteaId);
if (!giteaProvider) {
throw new TRPCError({
code: "NOT_FOUND",
message: "Gitea provider not found in the database",
});
command += `echo "❌ [ERROR] Gitea provider not found in the database"; exit 1;`;
return command;
}
const basePath = isCompose ? COMPOSE_PATH : APPLICATIONS_PATH;
const basePath = type === "compose" ? COMPOSE_PATH : APPLICATIONS_PATH;
const outputPath = join(basePath, appName, "code");
await recreateDirectory(outputPath);
command += `rm -rf ${outputPath};`;
command += `mkdir -p ${outputPath};`;
const repoClone = `${giteaOwner}/${giteaRepository}.git`;
const cloneUrl = buildGiteaCloneUrl(
@@ -229,134 +172,9 @@ export const cloneGiteaRepository = async (
giteaRepository!,
);
writeStream.write(`\nCloning Repo ${repoClone} to ${outputPath}...\n`);
try {
await spawnAsync(
"git",
[
"clone",
"--branch",
giteaBranch!,
"--depth",
"1",
...(enableSubmodules ? ["--recurse-submodules"] : []),
cloneUrl,
outputPath,
"--progress",
],
(data) => {
if (writeStream.writable) {
writeStream.write(data);
}
},
);
writeStream.write(`\nCloned ${repoClone}: ✅\n`);
} catch (error) {
writeStream.write(`ERROR Cloning: ${error}: ❌`);
throw error;
} finally {
writeStream.end();
}
};
export const cloneRawGiteaRepository = async (entity: Compose) => {
const {
appName,
giteaRepository,
giteaOwner,
giteaBranch,
giteaId,
enableSubmodules,
} = entity;
const { COMPOSE_PATH } = paths();
if (!giteaId) {
throw new TRPCError({
code: "NOT_FOUND",
message: "Gitea Provider not found",
});
}
await refreshGiteaToken(giteaId);
const giteaProvider = await findGiteaById(giteaId);
if (!giteaProvider) {
throw new TRPCError({
code: "NOT_FOUND",
message: "Gitea provider not found in the database",
});
}
const basePath = COMPOSE_PATH;
const outputPath = join(basePath, appName, "code");
await recreateDirectory(outputPath);
const cloneUrl = buildGiteaCloneUrl(
giteaProvider.giteaUrl,
giteaProvider.accessToken!,
giteaOwner!,
giteaRepository!,
);
try {
await spawnAsync("git", [
"clone",
"--branch",
giteaBranch!,
"--depth",
"1",
...(enableSubmodules ? ["--recurse-submodules"] : []),
cloneUrl,
outputPath,
"--progress",
]);
} catch (error) {
throw error;
}
};
export const cloneRawGiteaRepositoryRemote = async (compose: Compose) => {
const {
appName,
giteaRepository,
giteaOwner,
giteaBranch,
giteaId,
serverId,
enableSubmodules,
} = compose;
if (!serverId) {
throw new TRPCError({
code: "NOT_FOUND",
message: "Server not found",
});
}
if (!giteaId) {
throw new TRPCError({
code: "NOT_FOUND",
message: "Gitea Provider not found",
});
}
const { COMPOSE_PATH } = paths(true);
const giteaProvider = await findGiteaById(giteaId);
const basePath = COMPOSE_PATH;
const outputPath = join(basePath, appName, "code");
const cloneUrl = buildGiteaCloneUrl(
giteaProvider.giteaUrl,
giteaProvider.accessToken!,
giteaOwner!,
giteaRepository!,
);
try {
const command = `
rm -rf ${outputPath};
git clone --branch ${giteaBranch} --depth 1 ${enableSubmodules ? "--recurse-submodules" : ""} ${cloneUrl} ${outputPath}
`;
await execAsyncRemote(serverId, command);
} catch (error) {
throw error;
}
command += `echo "Cloning Repo ${repoClone} to ${outputPath}: ✅";`;
command += `git clone --branch ${giteaBranch} --depth 1 ${enableSubmodules ? "--recurse-submodules" : ""} ${cloneUrl} ${outputPath} --progress;`;
return command;
};
export const haveGiteaRequirements = (giteaProvider: Gitea) => {

View File

@@ -1,16 +1,11 @@
import { createWriteStream } from "node:fs";
import { join } from "node:path";
import { paths } from "@dokploy/server/constants";
import type { apiFindGithubBranches } from "@dokploy/server/db/schema";
import type { Compose } from "@dokploy/server/services/compose";
import { findGithubById, type Github } from "@dokploy/server/services/github";
import type { InferResultType } from "@dokploy/server/types/with";
import { createAppAuth } from "@octokit/auth-app";
import { TRPCError } from "@trpc/server";
import { Octokit } from "octokit";
import { recreateDirectory } from "../filesystem/directory";
import { execAsyncRemote } from "../process/execAsync";
import { spawnAsync } from "../process/spawnAsync";
export const authGithub = (githubProvider: Github): Octokit => {
if (!haveGithubRequirements(githubProvider)) {
@@ -123,42 +118,39 @@ interface CloneGithubRepository {
branch: string | null;
githubId: string | null;
repository: string | null;
logPath: string;
type?: "application" | "compose";
enableSubmodules: boolean;
serverId: string | null;
}
export const cloneGithubRepository = async ({
logPath,
type = "application",
...entity
}: CloneGithubRepository) => {
let command = "set -e;";
const isCompose = type === "compose";
const { APPLICATIONS_PATH, COMPOSE_PATH } = paths();
const writeStream = createWriteStream(logPath, { flags: "a" });
const { appName, repository, owner, branch, githubId, enableSubmodules } =
entity;
const {
appName,
repository,
owner,
branch,
githubId,
enableSubmodules,
serverId,
} = entity;
const { APPLICATIONS_PATH, COMPOSE_PATH } = paths(!!serverId);
if (!githubId) {
throw new TRPCError({
code: "NOT_FOUND",
message: "GitHub Provider not found",
});
command += `echo "Error: ❌ Github Provider not found"; exit 1;`;
return command;
}
const requirements = getErrorCloneRequirements(entity);
// Check if requirements are met
if (requirements.length > 0) {
writeStream.write(
`\nGitHub Repository configuration failed for application: ${appName}\n`,
);
writeStream.write("Reasons:\n");
writeStream.write(requirements.join("\n"));
writeStream.end();
throw new TRPCError({
code: "BAD_REQUEST",
message: "Error: GitHub repository information is incomplete.",
});
command += `echo "GitHub Repository configuration failed for application: ${appName}"; echo "Reasons:"; echo "${requirements.join("\n")}"; exit 1;`;
return command;
}
const githubProvider = await findGithubById(githubId);
@@ -167,193 +159,15 @@ export const cloneGithubRepository = async ({
const octokit = authGithub(githubProvider);
const token = await getGithubToken(octokit);
const repoclone = `github.com/${owner}/${repository}.git`;
await recreateDirectory(outputPath);
// await recreateDirectory(outputPath);
command += `rm -rf ${outputPath};`;
command += `mkdir -p ${outputPath};`;
const cloneUrl = `https://oauth2:${token}@${repoclone}`;
try {
writeStream.write(`\nCloning Repo ${repoclone} to ${outputPath}: ✅\n`);
const cloneArgs = [
"clone",
"--branch",
branch!,
"--depth",
"1",
...(enableSubmodules ? ["--recurse-submodules"] : []),
cloneUrl,
outputPath,
"--progress",
];
command += `echo "Cloning Repo ${repoclone} to ${outputPath}: ✅";`;
command += `git clone --branch ${branch} --depth 1 ${enableSubmodules ? "--recurse-submodules" : ""} ${cloneUrl} ${outputPath} --progress;`;
await spawnAsync("git", cloneArgs, (data) => {
if (writeStream.writable) {
writeStream.write(data);
}
});
writeStream.write(`\nCloned ${repoclone}: ✅\n`);
} catch (error) {
writeStream.write(`ERROR Cloning: ${error}: ❌`);
throw error;
} finally {
writeStream.end();
}
};
export const getGithubCloneCommand = async ({
logPath,
type = "application",
...entity
}: CloneGithubRepository & { serverId: string }) => {
const {
appName,
repository,
owner,
branch,
githubId,
serverId,
enableSubmodules,
} = entity;
const isCompose = type === "compose";
if (!serverId) {
throw new TRPCError({
code: "NOT_FOUND",
message: "Server not found",
});
}
if (!githubId) {
const command = `
echo "Error: ❌ Github Provider not found" >> ${logPath};
exit 1;
`;
await execAsyncRemote(serverId, command);
throw new TRPCError({
code: "NOT_FOUND",
message: "GitHub Provider not found",
});
}
const requirements = getErrorCloneRequirements(entity);
// Build log messages
let logMessages = "";
if (requirements.length > 0) {
logMessages += `\nGitHub Repository configuration failed for application: ${appName}\n`;
logMessages += "Reasons:\n";
logMessages += requirements.join("\n");
const escapedLogMessages = logMessages
.replace(/\\/g, "\\\\")
.replace(/"/g, '\\"')
.replace(/\n/g, "\\n");
const bashCommand = `
echo "${escapedLogMessages}" >> ${logPath};
exit 1; # Exit with error code
`;
await execAsyncRemote(serverId, bashCommand);
return;
}
const { COMPOSE_PATH, APPLICATIONS_PATH } = paths(true);
const githubProvider = await findGithubById(githubId);
const basePath = isCompose ? COMPOSE_PATH : APPLICATIONS_PATH;
const outputPath = join(basePath, appName, "code");
const octokit = authGithub(githubProvider);
const token = await getGithubToken(octokit);
const repoclone = `github.com/${owner}/${repository}.git`;
const cloneUrl = `https://oauth2:${token}@${repoclone}`;
const cloneCommand = `
rm -rf ${outputPath};
mkdir -p ${outputPath};
if ! git clone --branch ${branch} --depth 1 ${enableSubmodules ? "--recurse-submodules" : ""} --progress ${cloneUrl} ${outputPath} >> ${logPath} 2>&1; then
echo "❌ [ERROR] Fail to clone repository ${repoclone}" >> ${logPath};
exit 1;
fi
echo "Cloned ${repoclone} to ${outputPath}: ✅" >> ${logPath};
`;
return cloneCommand;
};
export const cloneRawGithubRepository = async (entity: Compose) => {
const { appName, repository, owner, branch, githubId, enableSubmodules } =
entity;
if (!githubId) {
throw new TRPCError({
code: "NOT_FOUND",
message: "GitHub Provider not found",
});
}
const { COMPOSE_PATH } = paths();
const githubProvider = await findGithubById(githubId);
const basePath = COMPOSE_PATH;
const outputPath = join(basePath, appName, "code");
const octokit = authGithub(githubProvider);
const token = await getGithubToken(octokit);
const repoclone = `github.com/${owner}/${repository}.git`;
await recreateDirectory(outputPath);
const cloneUrl = `https://oauth2:${token}@${repoclone}`;
try {
const cloneArgs = [
"clone",
"--branch",
branch!,
"--depth",
"1",
...(enableSubmodules ? ["--recurse-submodules"] : []),
cloneUrl,
outputPath,
"--progress",
];
await spawnAsync("git", cloneArgs);
} catch (error) {
throw error;
}
};
export const cloneRawGithubRepositoryRemote = async (compose: Compose) => {
const {
appName,
repository,
owner,
branch,
githubId,
serverId,
enableSubmodules,
} = compose;
if (!serverId) {
throw new TRPCError({
code: "NOT_FOUND",
message: "Server not found",
});
}
if (!githubId) {
throw new TRPCError({
code: "NOT_FOUND",
message: "GitHub Provider not found",
});
}
const { COMPOSE_PATH } = paths(true);
const githubProvider = await findGithubById(githubId);
const basePath = COMPOSE_PATH;
const outputPath = join(basePath, appName, "code");
const octokit = authGithub(githubProvider);
const token = await getGithubToken(octokit);
const repoclone = `github.com/${owner}/${repository}.git`;
const cloneUrl = `https://oauth2:${token}@${repoclone}`;
try {
const command = `
rm -rf ${outputPath};
git clone --branch ${branch} --depth 1 ${enableSubmodules ? "--recurse-submodules" : ""} ${cloneUrl} ${outputPath}
`;
await execAsyncRemote(serverId, command);
} catch (error) {
throw error;
}
return command;
};
export const getGithubRepositories = async (githubId?: string) => {

View File

@@ -1,8 +1,6 @@
import { createWriteStream } from "node:fs";
import { join } from "node:path";
import { paths } from "@dokploy/server/constants";
import type { apiGitlabTestConnection } from "@dokploy/server/db/schema";
import type { Compose } from "@dokploy/server/services/compose";
import {
findGitlabById,
type Gitlab,
@@ -10,9 +8,6 @@ import {
} from "@dokploy/server/services/gitlab";
import type { InferResultType } from "@dokploy/server/types/with";
import { TRPCError } from "@trpc/server";
import { recreateDirectory } from "../filesystem/directory";
import { execAsyncRemote } from "../process/execAsync";
import { spawnAsync } from "../process/spawnAsync";
export const refreshGitlabToken = async (gitlabProviderId: string) => {
const gitlabProvider = await findGitlabById(gitlabProviderId);
@@ -102,25 +97,34 @@ const getGitlabCloneUrl = (gitlab: GitlabInfo, repoClone: string) => {
return cloneUrl;
};
export const cloneGitlabRepository = async (
entity: ApplicationWithGitlab | ComposeWithGitlab,
logPath: string,
isCompose = false,
) => {
const writeStream = createWriteStream(logPath, { flags: "a" });
interface CloneGitlabRepository {
appName: string;
gitlabBranch: string | null;
gitlabId: string | null;
gitlabPathNamespace: string | null;
enableSubmodules: boolean;
serverId: string | null;
type?: "application" | "compose";
}
export const cloneGitlabRepository = async ({
type = "application",
...entity
}: CloneGitlabRepository) => {
let command = "set -e;";
const {
appName,
gitlabBranch,
gitlabId,
gitlabPathNamespace,
enableSubmodules,
serverId,
} = entity;
const { COMPOSE_PATH, APPLICATIONS_PATH } = paths(!!serverId);
if (!gitlabId) {
throw new TRPCError({
code: "NOT_FOUND",
message: "Gitlab Provider not found",
});
command += `echo "Error: ❌ Gitlab Provider not found"; exit 1;`;
return command;
}
await refreshGitlabToken(gitlabId);
@@ -130,127 +134,19 @@ export const cloneGitlabRepository = async (
// Check if requirements are met
if (requirements.length > 0) {
writeStream.write(
`\nGitLab Repository configuration failed for application: ${appName}\n`,
);
writeStream.write("Reasons:\n");
writeStream.write(requirements.join("\n"));
writeStream.end();
throw new TRPCError({
code: "BAD_REQUEST",
message: "Error: GitLab repository information is incomplete.",
});
command += `echo "❌ [ERROR] GitLab Repository configuration failed for application: ${appName}"; echo "Reasons:"; echo "${requirements.join("\n")}"; exit 1;`;
return command;
}
const { COMPOSE_PATH, APPLICATIONS_PATH } = paths();
const basePath = isCompose ? COMPOSE_PATH : APPLICATIONS_PATH;
const basePath = type === "compose" ? COMPOSE_PATH : APPLICATIONS_PATH;
const outputPath = join(basePath, appName, "code");
await recreateDirectory(outputPath);
command += `rm -rf ${outputPath};`;
command += `mkdir -p ${outputPath};`;
const repoClone = getGitlabRepoClone(gitlab, gitlabPathNamespace);
const cloneUrl = getGitlabCloneUrl(gitlab, repoClone);
try {
writeStream.write(`\nCloning Repo ${repoClone} to ${outputPath}: ✅\n`);
const cloneArgs = [
"clone",
"--branch",
gitlabBranch!,
"--depth",
"1",
...(enableSubmodules ? ["--recurse-submodules"] : []),
cloneUrl,
outputPath,
"--progress",
];
await spawnAsync("git", cloneArgs, (data) => {
if (writeStream.writable) {
writeStream.write(data);
}
});
writeStream.write(`\nCloned ${repoClone}: ✅\n`);
} catch (error) {
writeStream.write(`ERROR Cloning: ${error}: ❌`);
throw error;
} finally {
writeStream.end();
}
};
export const getGitlabCloneCommand = async (
entity: ApplicationWithGitlab | ComposeWithGitlab,
logPath: string,
isCompose = false,
) => {
const {
appName,
gitlabPathNamespace,
gitlabBranch,
gitlabId,
serverId,
enableSubmodules,
} = entity;
if (!serverId) {
throw new TRPCError({
code: "NOT_FOUND",
message: "Server not found",
});
}
if (!gitlabId) {
const command = `
echo "Error: ❌ Gitlab Provider not found" >> ${logPath};
exit 1;
`;
await execAsyncRemote(serverId, command);
throw new TRPCError({
code: "NOT_FOUND",
message: "Gitlab Provider not found",
});
}
const requirements = getErrorCloneRequirements(entity);
// Build log messages
let logMessages = "";
if (requirements.length > 0) {
logMessages += `\nGitLab Repository configuration failed for application: ${appName}\n`;
logMessages += "Reasons:\n";
logMessages += requirements.join("\n");
const escapedLogMessages = logMessages
.replace(/\\/g, "\\\\")
.replace(/"/g, '\\"')
.replace(/\n/g, "\\n");
const bashCommand = `
echo "${escapedLogMessages}" >> ${logPath};
exit 1; # Exit with error code
`;
await execAsyncRemote(serverId, bashCommand);
return;
}
const { COMPOSE_PATH, APPLICATIONS_PATH } = paths(true);
await refreshGitlabToken(gitlabId);
const gitlab = await findGitlabById(gitlabId);
const basePath = isCompose ? COMPOSE_PATH : APPLICATIONS_PATH;
const outputPath = join(basePath, appName, "code");
await recreateDirectory(outputPath);
const repoClone = getGitlabRepoClone(gitlab, gitlabPathNamespace);
const cloneUrl = getGitlabCloneUrl(gitlab, repoClone);
const cloneCommand = `
rm -rf ${outputPath};
mkdir -p ${outputPath};
if ! git clone --branch ${gitlabBranch} --depth 1 ${enableSubmodules ? "--recurse-submodules" : ""} --progress ${cloneUrl} ${outputPath} >> ${logPath} 2>&1; then
echo "❌ [ERROR] Fail to clone the repository ${repoClone}" >> ${logPath};
exit 1;
fi
echo "Cloned ${repoClone} to ${outputPath}: ✅" >> ${logPath};
`;
return cloneCommand;
command += `echo "Cloning Repo ${repoClone} to ${outputPath}: ✅";`;
command += `git clone --branch ${gitlabBranch} --depth 1 ${enableSubmodules ? "--recurse-submodules" : ""} ${cloneUrl} ${outputPath} --progress;`;
return command;
};
export const getGitlabRepositories = async (gitlabId?: string) => {
@@ -355,88 +251,6 @@ export const getGitlabBranches = async (input: {
}[];
};
export const cloneRawGitlabRepository = async (entity: Compose) => {
const {
appName,
gitlabBranch,
gitlabId,
gitlabPathNamespace,
enableSubmodules,
} = entity;
if (!gitlabId) {
throw new TRPCError({
code: "NOT_FOUND",
message: "Gitlab Provider not found",
});
}
const { COMPOSE_PATH } = paths();
await refreshGitlabToken(gitlabId);
const gitlabProvider = await findGitlabById(gitlabId);
const basePath = COMPOSE_PATH;
const outputPath = join(basePath, appName, "code");
await recreateDirectory(outputPath);
const repoClone = getGitlabRepoClone(gitlabProvider, gitlabPathNamespace);
const cloneUrl = getGitlabCloneUrl(gitlabProvider, repoClone);
try {
const cloneArgs = [
"clone",
"--branch",
gitlabBranch!,
"--depth",
"1",
...(enableSubmodules ? ["--recurse-submodules"] : []),
cloneUrl,
outputPath,
"--progress",
];
await spawnAsync("git", cloneArgs);
} catch (error) {
throw error;
}
};
export const cloneRawGitlabRepositoryRemote = async (compose: Compose) => {
const {
appName,
gitlabPathNamespace,
gitlabBranch,
gitlabId,
serverId,
enableSubmodules,
} = compose;
if (!serverId) {
throw new TRPCError({
code: "NOT_FOUND",
message: "Server not found",
});
}
if (!gitlabId) {
throw new TRPCError({
code: "NOT_FOUND",
message: "Gitlab Provider not found",
});
}
const { COMPOSE_PATH } = paths(true);
await refreshGitlabToken(gitlabId);
const gitlabProvider = await findGitlabById(gitlabId);
const basePath = COMPOSE_PATH;
const outputPath = join(basePath, appName, "code");
const repoClone = getGitlabRepoClone(gitlabProvider, gitlabPathNamespace);
const cloneUrl = getGitlabCloneUrl(gitlabProvider, repoClone);
try {
const command = `
rm -rf ${outputPath};
git clone --branch ${gitlabBranch} --depth 1 ${enableSubmodules ? "--recurse-submodules" : ""} ${cloneUrl} ${outputPath}
`;
await execAsyncRemote(serverId, command);
} catch (error) {
throw error;
}
};
export const testGitlabConnection = async (
input: typeof apiGitlabTestConnection._type,
) => {

Some files were not shown because too many files have changed in this diff Show More