django项目版本升级时触发migrate数据库版本更新

一、背景

• 考虑更新应用版本时同时对数据库进行迁移

• 对数据库连接地址等关键字串进行变量化

• 以下操作均在k8s环境先实现

二、django配置文件调整

分测试环境与生产环境

2.1 创建settings包

# 将settings.py文件移入到该目录并命名为base.py

2.2 修改manage.py文件

os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'website5.settings') 修改为
    profile = os.environ.get("PROJECT_PROFILE", "dev")
    if profile:
        os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'website5.settings.%s' % profile)
    else:
        os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'website5.settings.dev')

# 备注：这里目前考虑使用环境变量的方式去区分不同的应用配置文件

2.3 新建dev.py并将数据库连接信息剪切走

from .base import *

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.sqlite3',
        'NAME': BASE_DIR / 'db.sqlite3',
    }
}

2.4 修改base.py

BASE_DIR = Path(__file__).resolve().parent.parent
改为：
BASE_DIR = Path(__file__).resolve().parent.parent.parent

2.5 pycharm启动dev.py测试

# 备注：同时manage.py文件也需要调整并且加上对应的.dev（参考步骤2.2），方便后续的migrate操作

访问：http://127.0.0.1:8000/v1/articles/

2.6 配置生产prod.py

from .base import *

DATABASES = {
    "default": {
        "ENGINE": "django.db.backends.mysql",
        "OPTIONS": {
            "read_default_file": "/path/to/my.cnf",
        },
    }
}

# 备注：因为要考虑全局变量，这里采用文件方式进行映射。参考地址：https://docs.djangoproject.com/zh-hans/4.2/ref/databases/

2.7 生成requirements.txt文件

pip freeze > requirements.txt

# 备注：安装对应的pip模块

2.8 编写Dockerfile

[root@ip-172-20-21-242 website5]# cat Dockerfile 
FROM python:3.7.9

ARG REGISTRY=https://pypi.org/simple

RUN apt-get update \
    && apt-get install -y --no-install-recommends \
        postgresql-client \
    && rm -rf /var/lib/apt/lists/*

WORKDIR /app
COPY requirements.txt ./
RUN python -m pip install --upgrade pip && \
    pip install -i ${REGISTRY} -r requirements.txt
COPY . .

EXPOSE 8000
CMD ["python", "manage.py", "runserver", "--settings=mysite.settings.prod 0.0.0.0:8000"]

# 备注：dockerfile参考地址：https://hub.docker.com/_/django，该文件如果调试成功后需要放入到站点根目录下

三、Dockerfile打包镜像

3.1 build镜像

[root@ip-172-20-21-242 website5]# docker build -t registry.cn-hangzhou.aliyuncs.com/xiangys0134/django-api:v1.0.1 .

[root@ip-172-20-21-242 website5]# docker push registry.cn-hangzhou.aliyuncs.com/xiangys0134/django-api:v1.0.1

四、helm包配置

[k8s-dev-test@rancher-k8s-conn django-api]$ cat templates/deployment.yaml 
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    {{- if .Values.istio.enabled }}
    app: django-api-istio
    {{- with .Values.istio.versions }}
    {{- toYaml . | nindent 4 }}
    {{- end }}
    {{- else }}
    app: {{ template "django-api.fullname" . }}
    {{- end }}
  name: {{ include "django-api.fullname" . }}
spec:
  replicas: {{ .Values.replicaCount }}
  selector:
    matchLabels:
      {{- if .Values.istio.enabled }}
      app: django-api-istio
      {{- with .Values.istio.versions }}
      {{- toYaml . | nindent 6 }}
      {{- end }}
      {{- else }}
      app: {{ template "django-api.fullname" . }}
      {{- end }}
  template:
    metadata:
      labels:
        {{- if .Values.istio.enabled }}
        app: django-api-istio
        sidecar.istio.io/inject: "true"
        {{- with .Values.istio.versions }}
        {{- toYaml . | nindent 8 }}
        {{- end }}
        {{- else }}
        sidecar.istio.io/inject: "false"
        app: {{ template "django-api.fullname" . }}
        {{- end }}
    spec:
      volumes:
      - name: config-volume
        configMap:
          name: {{ template "django-api.fullname" . }}-cf
      {{- if .Values.skywalking.enabled }}
      - name: skywalking-agent
        emptyDir: { }
      {{- end }}
      {{- if .Values.skywalking.enabled }}
      initContainers:
        - name: agent-container
          #image: apache/skywalking-java-agent:8.5.0-alpine
          image: {{ .Values.skywalking.image }}:{{ .Values.skywalking.tag }}
          volumeMounts:
            - name: skywalking-agent
              mountPath: /agent
          command: [ "/bin/sh" ]
          args: [ "-c", "cp -R /skywalking/agent /agent/" ]
      {{- end }}
      containers:
        - command:
            - sh
            - "{{ .Values.command }}"
          envFrom:
          - configMapRef:
              name: {{ template "django-api.fullname" . }}
          image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"
          imagePullPolicy: {{ .Values.image.pullPolicy }}
          {{- if .Values.livenessProbe.enabled }}
          livenessProbe:
            failureThreshold: {{ .Values.livenessProbe.failureThreshold }}
            {{- if eq .Values.livenessProbe.type "tcpSocket" }}
            tcpSocket:
              port: {{ .Values.service.targetport }}
            {{- end }}
            {{- if eq .Values.livenessProbe.type "httpGet" }}
            httpGet:
              path: {{ .Values.livenessProbe.path }}
              port: {{ .Values.service.targetport }}
              scheme: HTTP
            {{- end }}
            initialDelaySeconds: {{ .Values.livenessProbe.initialDelaySeconds }}
            periodSeconds: {{ .Values.livenessProbe.periodSeconds }}
            successThreshold: {{ .Values.livenessProbe.successThreshold }}
            timeoutSeconds: {{ .Values.livenessProbe.timeoutSeconds }}
          {{- end }}
          name: {{ .Chart.Name }}
          {{- if .Values.readinessProbe.enabled }}
          readinessProbe:
            failureThreshold: {{ .Values.readinessProbe.failureThreshold }}
            {{- if eq .Values.readinessProbe.type "tcpSocket" }}
            tcpSocket:
              port: {{ .Values.service.targetport }}
            {{- end }}
            {{- if eq .Values.readinessProbe.type "httpGet" }}
            httpGet:
              path: {{ .Values.readinessProbe.path }}
              port: {{ .Values.service.targetport }}
              scheme: HTTP
            {{- end }}
            initialDelaySeconds: {{ .Values.readinessProbe.initialDelaySeconds }}
            periodSeconds: {{ .Values.readinessProbe.periodSeconds }}
            successThreshold: {{ .Values.readinessProbe.successThreshold }}
            timeoutSeconds: {{ .Values.readinessProbe.timeoutSeconds }}
          {{- end }}
          {{- with .Values.lifecycle }}
          lifecycle: 
            {{- toYaml . | nindent 16 }}
          {{- end }}
          ports:
          - name: http
            containerPort: {{ .Values.service.targetport }}
          volumeMounts:
            - name: config-volume
              mountPath: /path/to
          {{- if .Values.skywalking.enabled }}
            - name: skywalking-agent
              mountPath: /skywalking
          {{- end }}
          resources:
            limits:
              cpu: {{ .Values.resources.limits.cpu }}
              memory: {{ .Values.resources.limits.memory }}
            requests:
              cpu: {{ .Values.resources.requests.cpu }}
              memory: {{ .Values.resources.requests.memory }}
      #imagePullSecrets:
      #  - name: {{ template "django-api.fullname" . }}
      restartPolicy: Always
      {{- with .Values.nodeSelector }}
      nodeSelector:
        {{- toYaml . | nindent 8 }}
      {{- end }}
      {{- with .Values.affinity }}
      affinity:
        {{- toYaml . | nindent 8 }}
      {{- end }}
      {{- with .Values.tolerations }}
      tolerations:
        {{- toYaml . | nindent 8 }}
      {{- end }}

参考文档：https://gitee.com/xiangys0134/deploy/blob/master/django/%E9%A1%B9%E7%9B%AE%E7%AE%A1%E7%90%86/helm%E5%8C%85/django-api/templates/deployment.yaml

五、部署服务

5.1 配置mysql连接

[k8s-dev-test@rancher-k8s-conn django-api]$ vim values.yaml
mysql:
  host: "192.168.7.225"
  database: "django_api_test"
  user: "ops_test"
  password: "ops_test"
  port: 3306
  character: "utf8"

MySQL [(none)]> create database django_api_test;        # mysql创建该库

5.2 配置启动命令

[k8s-dev-test@rancher-k8s-conn django-api]$ vim values.yaml

备注：migrate表示服务启动后同时更新数据库版本

5.3 部署服务

[k8s-dev-test@rancher-k8s-conn django-api]helm upgrade -i django-api .
[k8s-dev-test@rancher-k8s-conn django-api] kubectl get pods |grep django-api

5.4 查看数据库版本是否更新

5.5 查看pod日志

[k8s-dev-test@rancher-k8s-conn django-api]$ kubectl logs django-api-64cbdb4675-wnb5s

六、测试数据及接口

6.1 进入容器

[k8s-dev-test@rancher-k8s-conn django-api]$ kubectl exec -it django-api-64cbdb4675-wnb5s bash
root@django-api-64cbdb4675-wnb5s:/app# printenv |grep -i project_profile

root@django-api-64cbdb4675-wnb5s:/app# cat manage.py

6.2 创建超级用户

root@django-api-64cbdb4675-wnb5s:/app# python manage.py createsuperuser

6.3 后台登陆并添加数据

http://192.168.7.41:30284/admin

6.4 查看接口

http://192.168.7.41:30284/v1/articles/

七、测试新版本上线

7.1 添加对应model对象

class Article3(models.Model):
    title = models.CharField(max_length=256)
    body = models.TextField()
    liked_by = models.ManyToManyField(to=User)

    def __str__(self):
        return self.title

7.2 添加对应view功能

class ArticleList3(APIView):
    """ 查看列表和创建一个新对象"""
    # authentication_classes = (TokenAuthentication, )
    # permission_classes = (IsAuthenticated,)

    def get(self, request, format=None):
        articles = Article3.objects.all()
        serializer = Article3ModelSerializer(articles, many=True)
        return Response(serializer.data)

    def post(self, request, format=None):
        serializer = Article3ModelSerializer(data=request.data)
        if serializer.is_valid():
            serializer.save(author=request.user)
            return Response(serializer.data, status=status.HTTP_201_CREATED)
        return Response(serializer.errors, status=status.HTTP_400_BAD_REQUEST)

    # 备注：因篇幅原因省略掉了序列化器对象

7.3 url配置

re_path(r'^articles3/$', views.ArticleList3.as_view()),

7.4 测试环境测试

pycharm 执行如下命令，这里选择的是测试环境
# python manage.py makemigrations
# python manage.py migrate

7.5 查看测试环境数据库表信息

7.6 推送代码，并对新版本进行打包

[root@ip-172-20-21-242 website5]# docker build -t registry.cn-hangzhou.aliyuncs.com/xiangys0134/django-api:v2.0.2 .
[root@ip-172-20-21-242 website5]# docker push registry.cn-hangzhou.aliyuncs.com/xiangys0134/django-api:v2.0.2

7.7 部署新版本

[k8s-dev-test@rancher-k8s-conn django-api]$ vim values.yaml
image:
  ...
  tag: "v2.0.2"

7.8 部署

[k8s-dev-test@rancher-k8s-conn django-api]helm upgrade -i django-api .
[k8s-dev-test@rancher-k8s-conn django-api] kubectl get pods |grep django-api

[k8s-dev-test@rancher-k8s-conn django-api]$ kubectl logs django-api-5679bc695b-qqr7x

7.9 访问

http://192.168.7.41:30284/v1/articles3/

八、总结

这里主要考虑如何去区分生产环境以及测试环境。因以上步骤均通过手动命令执行，可以接入jenkins进行自动化打包以及自动化部署。

helm包参考地址：

https://gitee.com/xiangys0134/deploy/tree/master/django/%E9%A1%B9%E7%9B%AE%E7%AE%A1%E7%90%86/helm%E5%8C%85/django-api

django项目参考地址：

https://gitee.com/xiangys0134/deploy/tree/master/django/%E9%A1%B9%E7%9B%AE%E7%AE%A1%E7%90%86/%E4%BB%A3%E7%A0%81/website5