Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/unit-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -46,4 +46,4 @@ jobs:
- name: 🧪 Run Tests
run: |
pnpm -F "!@teable/backend" -r --parralel test-unit
pnpm -F "!@teable/backend" -r --parallel test-unit
25 changes: 20 additions & 5 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,25 +17,28 @@ corepack enable
pnpm install
```

### 2. Database Selection
We support SQLite (dev only) and PostgreSQL. PostgreSQL is recommended and requires Docker installed.
### 2. Database Setup

Teable uses PostgreSQL for development and requires Docker installed.

```bash
# Switch between SQLite and PostgreSQL
make switch-db-mode
```

### 3. Environment Configuration (Optional)

```bash
cd apps/nextjs-app
cp .env.development .env.development.local
```

### 4. Start Development Server

```bash
cd apps/nestjs-backend
pnpm dev
```

This will automatically start both backend and frontend servers with hot reload enabled.

## Continuous Development
Expand Down Expand Up @@ -64,17 +67,21 @@ Teable uses Prisma as ORM for database management. Follow these steps for schema
1. Modify `packages/db-main-prisma/prisma/template.prisma`

2. Generate Prisma schemas:

```bash
make gen-prisma-schema
```
This generates both SQLite and PostgreSQL schemas and TypeScript definitions.

This generates the PostgreSQL schema and TypeScript definitions.

3. Create migrations file:

```bash
make db-migration
```

4. Apply migrations:

```bash
make switch-db-mode
```
Expand All @@ -85,6 +92,7 @@ make switch-db-mode
## Testing

### E2E Tests

Located in `apps/nestjs-backend`:

```bash
Expand All @@ -99,6 +107,7 @@ pnpm test-e2e [test-file]
```

### Unit Tests

```bash
# Run all unit tests
pnpm g:test-unit
Expand All @@ -112,13 +121,16 @@ pnpm test-unit [test-file]
```

### IDE Integration

Using VSCode/Cursor:

1. For E2E tests in `apps/nestjs-backend`:

- Switch to test file (e.g. `apps/nestjs-backend/test/record.e2e-spec.ts`)
- Select "vitest e2e nest backend" in Debug panel

2. For unit tests in different packages:
- For `packages/core`:
- For `packages/core`:
- Switch to test file (e.g. `packages/core/src/utils/date.spec.ts`)
- Select "vitest core" in Debug panel
- For other packages, select their corresponding debug configuration
Expand All @@ -130,6 +142,7 @@ Each package has its own debug configuration in VSCode/Cursor, make sure to sele
This repo follows [conventional commit](https://www.conventionalcommits.org/en/v1.0.0/) format.

### Common Prefixes

- **feat**: New feature
- **fix**: Bug fix
- **docs**: Documentation changes
Expand All @@ -144,9 +157,11 @@ This repo follows [conventional commit](https://www.conventionalcommits.org/en/v
## Docker Build

### Building Images Locally

- `teable`: The main application image

#### Build the Application Image

> **Note**
> You should run this command in the root directory.

Expand Down
78 changes: 15 additions & 63 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -42,10 +42,8 @@ SCRATCH ?= /tmp

UNAME_S := $(shell uname -s)

# prisma database url defaults
SQLITE_PRISMA_DATABASE_URL ?= file:../../db/main.db
# set param statement_cache_size=1 to avoid query error `ERROR: cached plan must not change result type` after alter column type (modify field type)
POSTGES_PRISMA_DATABASE_URL ?= postgresql://teable:teable\@127.0.0.1:5432/teable?schema=public\&statement_cache_size=1
POSTGRES_PRISMA_DATABASE_URL ?= postgresql://teable:teable\@127.0.0.1:5432/teable?schema=public\&statement_cache_size=1

# If the first make argument is "start", "stop"...
ifeq (docker.start,$(firstword $(MAKECMDGOALS)))
Expand Down Expand Up @@ -112,17 +110,15 @@ DOCKER_COMPOSE_ARGS := DOCKER_UID=$(shell id -u) \

define print_db_mode_options
@echo -e "\nSelect a database to start.\n"
@echo -e "1)sqlite Lightweight embedded, ideal for mobile and embedded systems, simple, resource-efficient, easy integration (default database)"
@echo -e "2)postges(pg) Powerful and scalable, suitable for complex enterprise needs, highly customizable, rich community support\n"
@echo -e "1)postgres(pg) Powerful and scalable, suitable for complex enterprise needs, highly customizable, rich community support\n"
endef

define print_db_push_options
@echo -e "The 'db pull' command connects to your database and adds Prisma models to your Prisma schema that reflect the current database schema.\n"
@echo -e "1) sqlite"
@echo -e "2) postges(pg)\n"
@echo -e "1) postgres(pg)\n"
endef

.PHONY: db-mode sqlite.mode postgres.mode gen-prisma-schema gen-sqlite-prisma-schema gen-postgres-prisma-schema
.PHONY: db-mode postgres.mode gen-prisma-schema gen-postgres-prisma-schema
.DEFAULT_GOAL := help

docker.create.network:
Expand Down Expand Up @@ -199,13 +195,6 @@ build.db-migrate:
--tag=teable-db-migrate:develop


sqlite.integration.test:
@export PRISMA_DATABASE_URL='file:../../db/main.db'; \
export CALC_CHUNK_SIZE=400; \
make sqlite.mode; \
pnpm -F "./packages/**" run build; \
pnpm g:test-e2e-cover

postgres.integration.test: docker.create.network
@TEST_PG_CONTAINER_NAME=teable-postgres-$(CI_JOB_ID); \
docker rm -fv $$TEST_PG_CONTAINER_NAME | true; \
Expand All @@ -218,21 +207,12 @@ postgres.integration.test: docker.create.network
pnpm g:test-e2e-cover && \
docker rm -fv $$TEST_PG_CONTAINER_NAME

gen-sqlite-prisma-schema:
@cd ./packages/db-main-prisma; \
echo '{ "PRISMA_PROVIDER": "sqlite" }' | pnpm mustache - ./prisma/template.prisma > ./prisma/sqlite/schema.prisma
@echo 'generate【 prisma/sqlite/schema.prisma 】success.'

gen-postgres-prisma-schema:
@cd ./packages/db-main-prisma; \
echo '{ "PRISMA_PROVIDER": "postgres" }' | pnpm mustache - ./prisma/template.prisma > ./prisma/postgres/schema.prisma
@echo 'generate【 prisma/postgres/schema.prisma 】success.'

gen-prisma-schema: gen-sqlite-prisma-schema gen-postgres-prisma-schema ## Generate 'schema.prisma' files for all versions of the system

sqlite-db.push: ## db.push by sqlite
@cd ./packages/db-main-prisma; \
pnpm prisma-db-push --schema ./prisma/sqlite/schema.prisma
gen-prisma-schema: gen-postgres-prisma-schema ## Generate 'schema.prisma' files for all versions of the system

postgres-db.push: ## db.push by postgres
@cd ./packages/db-main-prisma; \
Expand All @@ -241,20 +221,11 @@ postgres-db.push: ## db.push by postgres
db.push: ## connects to your database and adds Prisma models to your Prisma schema that reflect the current database schema.
$(print_db_push_options)
@read -p "Enter a command: " command; \
if [ "$$command" = "1" ] || [ "$$command" = "sqlite" ]; then \
make gen-sqlite-prisma-schema; \
make sqlite-db.push; \
elif [ "$$command" = "2" ] || [ "$$command" = "postges" ] || [ "$$command" = "pg" ]; then \
if [ "$$command" = "1" ] || [ "$$command" = "postgres" ] || [ "$$command" = "pg" ]; then \
make gen-postgres-prisma-schema; \
make postgres-db.push; \
else echo "Unknown command."; fi

sqlite-db-migration:
@_MIGRATION_NAME=$(if $(_MIGRATION_NAME),$(_MIGRATION_NAME),`read -p "Enter name of the migration (sqlite): " migration_name; echo $$migration_name`); \
make gen-sqlite-prisma-schema; \
PRISMA_DATABASE_URL=file:../../db/.shadow/main.db \
pnpm -F @teable/db-main-prisma prisma-migrate dev --schema ./prisma/sqlite/schema.prisma --name $$_MIGRATION_NAME

postgres-db-migration:
@_MIGRATION_NAME=$(if $(_MIGRATION_NAME),$(_MIGRATION_NAME),`read -p "Enter name of the migration (postgres): " migration_name; echo $$migration_name`); \
make gen-postgres-prisma-schema; \
Expand All @@ -263,52 +234,33 @@ postgres-db-migration:

db-migration: ## Reruns the existing migration history in the shadow database in order to detect schema drift (edited or deleted migration file, or a manual changes to the database schema)
@read -p "Enter name of the migration: " migration_name; \
make sqlite-db-migration _MIGRATION_NAME=$$migration_name; \
make postgres-db-migration _MIGRATION_NAME=$$migration_name

sqlite.mode: ## sqlite.mode
@cd ./packages/db-main-prisma; \
pnpm prisma-generate --schema ./prisma/sqlite/schema.prisma; \
pnpm prisma-migrate deploy --schema ./prisma/sqlite/schema.prisma

postgres.mode: ## postgres.mode
@cd ./packages/db-main-prisma; \
pnpm prisma-generate --schema ./prisma/postgres/schema.prisma; \
pnpm prisma-migrate deploy --schema ./prisma/postgres/schema.prisma
# Override environment variable files based on variables
RUN_DB_MODE ?= sqlite
RUN_DB_MODE ?= postgres
FILE_ENV_PATHS = $(ENV_PATH)/.env.development* $(ENV_PATH)/.env.test*
switch.prisma.env:
ifeq ($(CI)-$(RUN_DB_MODE),0-sqlite)
@for file in $(FILE_ENV_PATHS); do \
echo $$file; \
perl -i -pe 's~^PRISMA_DATABASE_URL=.*~PRISMA_DATABASE_URL=$(SQLITE_PRISMA_DATABASE_URL)~' $$file; \
if ! grep -q '^CALC_CHUNK_SIZE=' $$file; then \
echo "CALC_CHUNK_SIZE=400" >> $$file; \
else \
perl -i -pe 's~^CALC_CHUNK_SIZE=.*~CALC_CHUNK_SIZE=400~' $$file; \
fi; \
done
else ifeq ($(CI)-$(RUN_DB_MODE),0-postges)
ifeq ($(CI)-$(RUN_DB_MODE),0-postgres)
@for file in $(FILE_ENV_PATHS); do \
echo $$file; \
perl -i -pe 's~^PRISMA_DATABASE_URL=.*~PRISMA_DATABASE_URL=$(POSTGES_PRISMA_DATABASE_URL)~' $$file; \
perl -i -pe 's~^PRISMA_DATABASE_URL=.*~PRISMA_DATABASE_URL=$(POSTGRES_PRISMA_DATABASE_URL)~' $$file; \
done
endif

switch-db-mode: ## Switch Database environment
$(print_db_mode_options)
@read -p "Enter a command: " command; \
if [ "$$command" = "1" ] || [ "$$command" = "sqlite" ]; then \
make switch.prisma.env RUN_DB_MODE=sqlite; \
make sqlite.mode; \
elif [ "$$command" = "2" ] || [ "$$command" = "postges" ] || [ "$$command" = "pg" ]; then \
make switch.prisma.env RUN_DB_MODE=postges; \
if [ "$$command" = "1" ] || [ "$$command" = "postgres" ] || [ "$$command" = "pg" ]; then \
make switch.prisma.env RUN_DB_MODE=postgres; \
make docker.up teable-postgres; \
make docker.await teable-postgres; \
make postgres.mode; \
else \
echo "Unknown command."; fi
make docker.await teable-postgres; \
make postgres.mode; \
else \
echo "Unknown command."; fi

help: ## show this help
@grep -E '^[a-zA-Z_-]+:.*?## .*$$' $(MAKEFILE_LIST) | sort | awk 'BEGIN {FS = ":.*?## "}; {printf "\033[36m%-30s\033[0m %s\n", $$1, $$2}'
8 changes: 3 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -156,7 +156,6 @@ These platforms are easy to deploy with one click and come with free credits.

[![Deploy on AlibabaCloud ComputeNest](https://service-info-public.oss-cn-hangzhou.aliyuncs.com/computenest-en.svg)](https://computenest.console.aliyun.com/service/instance/create/default?ServiceName=Teable%20%E7%A4%BE%E5%8C%BA%E7%89%88)


## Development

#### 1. Initialize
Expand All @@ -169,9 +168,7 @@ corepack enable
pnpm install
```

#### 2. Select Database

we currently support `sqlite` (dev only) and `postgres`, you can switch between them by running the following command
#### 2. Initialize Postgres

```sh
make switch-db-mode
Expand All @@ -194,6 +191,7 @@ pnpm dev
```

By default, the plugin development server is not started. To preview and develop plugins, run:

```sh
# build packages
pnpm build:packages
Expand All @@ -202,8 +200,8 @@ pnpm build:packages
cd plugins
pnpm dev
```
This will start the plugin development server on port 3002.

This will start the plugin development server on port 3002.

## Why Teable?

Expand Down
1 change: 1 addition & 0 deletions apps/nestjs-backend/src/configs/threshold.config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,7 @@ export const thresholdConfig = registerAs('threshold', () => ({
automation: {
maxEmailsPerPoll: Number(process.env.AUTOMATION_MAX_EMAILS_PER_POLL ?? 100),
maxEmailDedupWindowSize: Number(process.env.AUTOMATION_MAX_EMAIL_DEDUP_WINDOW_SIZE ?? 500),
httpRequestTimeout: Number(process.env.AUTOMATION_HTTP_REQUEST_TIMEOUT ?? 300_000), // 5 mins
},
}));

Expand Down
Loading
Loading