GET and PATCH requests on the Course model

The next two endpoints related to the Course model are the retrieve and update. In terms of retrieve, there will be two types of GET requests - fetch all courses and fetch a particular course. The results should also change according to who is making the requests. An admin user should be able to see all courses, including those in draft mode or those that have been archived. A normal user should only see those courses that have been published and are active.

I use the ListModelMixin and the RetrieveModelMixin  from Django Rest Framework to reduce the amount of code. In terms of fetching a particular course, this has to be done using the course slug rather than the ID as the slug is more human readable. All that needs to be done is to use the list() and retrieve() methods in the get() method:

def get(self, request, *args, **kwargs):
    '''Fetch all courses or specific course by slug'''
    slug = self.kwargs.get('slug', None)
    self.authenticate(request)
    if slug:
        return self.retrieve(request, *args, *kwargs)
    else:
        return self.list(request, *args, **kwargs)

These two methods under the hood call the get_queryset() method which is provided by GenericAPIView. The authenticate method is being called here to ensure that any user credentials in the header are extracted. The get_queryset method is:

def get_queryset(self, *args, **kwargs):
    '''Return courses not in draft mode and not archived'''
    if self.request.user is not None and self.request.user.is_staff:
        return Course.objects.all()
    return Course.objects.fetch_courses()

Here, is the logged in user is an admin, all courses will be returned or else a subset will be returned. This subset is determined by the method fetch_courses() which has been defined as a query in a model manager.

class CourseManager(models.Manager):
    '''Manager for Course model'''

    def fetch_courses(self, is_draft=False, is_archived=False, *args, **kwargs):
        return self.get_queryset().filter(
            is_draft=is_draft,
            is_archived=is_archived
        )

Here fetch_courses() has default arguments that courses in draft mode or those that have been archived should be excluded. The default get_queryset() is merely the all() filter. Therefore, fetch_courses() without any arguments will return those courses that have been published and have not been archived.

Coming to the last endpoint - the update course API call. For this the UpdateModelMixin is used with Django Rest Framework. The UpdateModelMixin provides an update() and a partial_update() method. The partial_update() method is suitable for PATCH requests with fields being optional. The patch() method is very similar to the create() method for POST requests:

def patch(self, request, *args, **kwargs):
    '''Update a course by an authenticated instructor'''
    try:
        self.authenticate(request)
        return self.partial_update(request, *args, **kwargs)
    except CourseGenericError as e:
        return Response(
            data=str(e),
            status=status.HTTP_400_BAD_REQUEST
        )
    except CourseForbiddenError as e:
        logger.critical('Course update by non admin attempted')
        return Response(
            data=str(e),
            status=status.HTTP_403_FORBIDDEN
        )
    except ValidationError as e:
        logger.error('Error updating course - {}'.format(str(e)))
        return rest_framework_validation_error(e, 'Course could not be updated')
    except InvalidToken as e:
        logger.critical('Course update by non admin attempted')
        return Response(
            data='Must be logged in as administrator to update a course',
            status=status.HTTP_403_FORBIDDEN
        )
    except Exception as e:
        logger.critical('Error updating course - {}'.format(str(e)))
        return Response(
            data=DEFAULT_ERROR_RESPONSE,
            status=status.HTTP_400_BAD_REQUEST
        )

This patch() method calls the partial_update() method similar to how the create() method call the perform_create() method.

def perform_update(self, serializer):
    '''Update a course by an authenticated instructor'''
    course = serializer.save(user=self.request.user)

By calling the save() method on the serializer, the update() method of the serializer will be called instead of the create() method:

def update(self, instance, validated_data):
    user = validated_data.get('user', None)
    if user is not None and instance.check_user_is_instructor(user):
        instance.title = validated_data.get('title', instance.title)
        instance.subtitle = validated_data.get(
            'subtitle', instance.subtitle)
        instance.description = validated_data.get(
            'description', instance.description)
        instance.price = validated_data.get('price', instance.price)
        instance.is_free = validated_data.get('is_free', instance.is_free)
        instance.is_published = validated_data.get(
            'is_published', instance.is_published)
        instance.is_draft = validated_data.get(
            'is_draft', instance.is_draft)
        instance.is_archived = validated_data.get(
            'is_archived', instance.is_archived)
        instance.save()
        return instance
    else:
        raise CourseForbiddenError(
            'Only an instructor of a course can update a course'
        )

Here, there is a model method check_user_is_instructor():

def check_user_is_instructor(self, user):
    '''Check if a user is an instructor of the course'''
    instuctor_emails = [x.username for x in self.instructors.all()]
    if user and user.username in instuctor_emails:
        return True
    return False

This is to ensure that only a user who is one of the instructors can update the course.

With this the CRU endpoints of CRUD for courses is done. The D (delete) endpoint will be deferred until I figure out the best kind of multi-factor authentication should be used when deleting data through API calls.

Detailed tests have been written for these three endpoints. The tests will be skipped in the blog posts as it is merely to verify the functionality.

Getting started with the Course model

Spent a couple of weeks coding the basics of a course after the basic user authentication was done. The plan is to have the outermost model to be that of a course which in turn contains lectures and the lectures will then have content which could be videos, documents or just text.

So far the idea is that a course can be created only by an admin user. This deviates from the normal online model where any instructor can create a course, as in the case of this web app, the idea is for the admin to be the creators and managers of courses. So a course will have the usual fields like title, subtitle, description and price. It will also have a few flags to indicate whether the course is free, in draft mode, is archived. The course model will link to the user model for the list of instructors and students.

class Course(models.Model):
    '''Course model'''
    title = models.CharField(max_length=300, unique=True)
    subtitle = models.CharField(max_length=300, null=True, blank=True)
    slug = models.SlugField(max_length=200)
    description = models.TextField()
    instructors = models.ManyToManyField(User, related_name='courses_taught')
    students = models.ManyToManyField(User, blank=True)
    price = models.DecimalField(default=10.99, max_digits=4, decimal_places=2)
    is_free = models.BooleanField(default=False)
    is_published = models.BooleanField(default=False)
    is_draft = models.BooleanField(default=True)
    is_archived = models.BooleanField(default=False)


def generate_course_slug(sender, instance, *args, **kwargs):
    '''Generate slug for course'''
    if not instance.slug:
        instance.slug = slugify(instance.title)


pre_save.connect(generate_course_slug, sender=Course)

The computed field is the slug for the Course which is merely to be able to access the course as a web link without using the course ID but rather a human readable string. Since the course title is unique, the slugify method available with Django does the job.

With the Course model defined, the next step is to define the CourseSerializer:

class CourseSerializer(serializers.ModelSerializer):
    '''Serializer for course'''

    title = serializers.CharField(
        error_messages={
            'blank': 'Course title is required',
            'required': 'Course title is required'
        },
        validators=[
            UniqueValidator(
                queryset=Course.objects.all(),
                message='A course with this title already exists'
            )
        ]
    )

    class Meta:
        model = Course
        fields = ['title', 'subtitle', 'description', 'price', 'is_free',]
        extra_kwargs = {
            'description': {
                'error_messages': {
                    'blank': 'Course description is required',
                    'required': 'Course description is required'
                }
            },
        }

In the most basic form, the serializer accepts the fields related to title, subtitle, description, price and whether the course is free. For the title, besides the required error, an additional validator needs to be included for checking whether the title is unique. This UniqueValidator needs a queryset against which it will check for uniqueness (which is all courses).

Added to these classes, a few more methods needed to be added. First, methods are needed for adding instructors and students to a course. These can be model methods. In the case of instructors, it is necessary to check if the users are admin.

def add_instructor(self, user):
    '''Add instructors to the course'''
    if user.is_staff:
        self.instructors.add(user)
    else:
        raise CourseForbiddenError('Instructors have to be administrators')

def add_students(self, user):
    '''Add students to the course'''
    self.students.add(user)

At the model level, it is necessary to add a check before the model is saved - if a course is free, the price should be 0, while if a course is not specified as free, a price should be specified. Model level validation can be done with the clean_fields() method, but this clean() method will be called when a ModelForm is saved and not when the model is created or saved elsewhere. Therefore, I overloaded the save() method in addition to creating a clean_fields() method which works with the admin dashboard:

def save(self, *args, **kwargs):
    if not self.is_free and self.price <= 0:
        raise CourseGenericError('Price of a non-free course is required.')
    if self.is_free:
        self.price = 0.00
    super().save(*args, **kwargs)

def clean_fields(self, exclude=None):
    '''Validation in admin dashboard'''
    if not self.is_free and self.price <= 0:
        raise ValidationError('Price of a non-free course is required.')

With this update to the Course model, the CourseSerializer can also be updated to introduce some validation and also facilitate the addition of instructors at the time of creating a course.

def validate(self, data):
    course_is_free = data.get('is_free', False)
    course_price = data.get('price', None)
    if course_price is not None and course_price > 0:
        data['is_free'] = False
    elif course_is_free:
        data['price'] = 0.00
    elif course_price is None or course_price <= 0:
        raise CourseGenericError('Course price is required')
    return data

def create(self, validated_data):
    user = validated_data.get('user', None)
    if user is not None and user.is_staff:
        del validated_data['user']
        course = Course.objects.create(**validated_data)
        course.add_instructor(user)
        return course
    else:
        raise CourseForbiddenError(
            'Must be logged in as administrator to create a course'
        )

The validation ensures that if the price is provided in the body of an API call, the is_free flag is automatically turned OFF. Alternatively, the API call can only mention that the course is free in which case the price needs to be set to zero. If nothing is mentioned about the course being free and the price is either skipped or is 0 or a negative number, an error should be thrown.

While creating a course, the user is extracted from the validated data and check is made whether this user exists and is admin. This is to ensure that an anonymous user or a normal non-admin user cannot create a course. The admin user is automatically added to the instructor list. This user needs to be passed into the save() method of the serializer, as any additional fields passed to the save method will show up in the validated_data dictionary.

With this, a basic course creation endpoint can be created and the following view class written that inherits the CreateAPIView from the generic view classes in Django Rest Framework:

class CourseView(
    CreateAPIView,
    UserAuthentication
):
    '''Create a course with logged-in user as instructor'''

    serializer_class = CourseSerializer
    user_model = User
    lookup_field = 'slug'
    queryset = Course.objects.all()

    def perform_create(self, serializer):
        '''Create a new course with authenticated user as instructor'''
        course = serializer.save(user=self.request.user)
        logger.info(
            'Creating course {course} by user {user}'.format(
                course=course.title,
                user=self.request.user.username
            )
        )

    def create(self, request, *args, **kwargs):
        '''Create a new course - POST request'''
        try:
            self.authenticate(request)
            return super().create(request, *args, **kwargs)
        except CourseGenericError as e:
            logger.error('Error creating course - {}'.format(str(e)))
            return Response(
                data=str(e),
                status=status.HTTP_400_BAD_REQUEST
            )
        except CourseForbiddenError as e:
            logger.critical('Course creation by non admin attempted')
            return Response(
                data=str(e),
                status=status.HTTP_403_FORBIDDEN
            )
        except ValidationError as e:
            logger.error('Error creating course - {}'.format(str(e)))
            return rest_framework_validation_error(e, 'Course could not be created')
        except InvalidToken as e:
            logger.critical('Course creation by non admin attempted')
            return Response(
                data='Must be logged in as administrator to create a course',
                status=status.HTTP_403_FORBIDDEN
            )
        except Exception as e:
            logger.critical('Error creating course - {}'.format(str(e)))
            return Response(
                data=DEFAULT_ERROR_RESPONSE,
                status=status.HTTP_400_BAD_REQUEST
            )

The CreateAPIView provides the create() method which essentially handles the POST request. The CreateAPIView inherits from the GenericAPIView which needs a serializer_class and queryset to be specified.

In order to handle authentication, I created a custom class UserAuthentication which inherits JWTAuthentication.

class UserAuthentication(JWTAuthentication):
    '''Returns user from token in header'''

    def authenticate(self, request, check_admin=True, *args, **kwargs):
        user = super().authenticate(request, *args, **kwargs)
        if user is not None:
            if check_admin and not user[0].is_staff:
                return None
            request.user = user[0]
            return user[0]

JWTAuthentication contains an authenticate method which extracts from the request a header with the name Authorization which is a string in the format "Bearer <jwt-access-token>". The good part is that the authenticate() method that ships with JWTAuthentication does this and returns a user object. The problem is that the object is not just the user object but also details about the token - when it was created and when it will expire. To further generalize this method, an optional argument check_admin is passed which is by default True - therefore this method will by default return a user object if the user is admin. Moreover, the user object is attached to the request object.

By inheriting this custom UserAuthentication class in the CourseView, an authenticate() method is available which will provide a user object if an admin user access token is provided in the header of the POST request.

To be able to use this user object while creating the course, it is necessary to overload the perform_create() method which is used by the create() method of CreateAPIView. By passing the user into the save() method, the user can be extracted from the validated data while saving the serializer.

With this POST requests to create new courses are in place and the C in CRUD is done. Remaining are RUD - Retrieve, Update and Delete. Out of these, Delete will be deferred until later, as this might need two-factor authentication.

Code refactor after manual security review

After performing security analysis using semgrep, I will perform a quick analysis of the code so far in order to think of possible vulnerabilities. Though it is still a rudimentary stage of the app, security if taken into consideration at the early stages might result in a more secure app.

I will use the Open Web Application Security Project (OWASP) and the vulnerabilities listed on their website to comb through the code. OWASP has a huge list of vulnerabilities listed on their website:

https://owasp.org/www-community/vulnerabilities/

They also have a shorter top 10 vulnerabilities list:

https://owasp.org/www-project-top-ten/

To begin with, one major shortcoming so far is the lack of any logs generated. Ideally, I would need to log every action with separate tags – info, debug, warning, error etc. These tags would help to filter the logs later and might also be used for generating automatic notifications in cases of serious breaches. Therefore, if there is an ongoing attack, rather than wait for the issue to be reported, if logs can be generated that send an automatic notification in terms of an email or a text message, this could help to minimize the damage or preventing it from completely bringing down the app. I will soon investigate logging with Django.

Now to analyze the code so far and examine ways in which it could be made a bit more secure. I will go through the outer view classes first. The first view class is the RegisterUserView. To begin with, since it is a POST request, this might need to handle CSRF. Since, there is no frontend at the moment, and requests are made with the browser using the forms generated by DRF or through a client such as Postman. Unless there is another browser accessing the API endpoints from another domain, CSRF does not really come into play. So this issue can be handled later for all endpoints.

In terms of the behaviour of this endpoint, for a valid registration that contains a username that is a genuine email and a password (not validated at the moment), the view returns an object with the username and the inactive status. This might be ok as the view is merely returning back what was supplied to it in the POST data. Now to look at when the API fails, this would occur when either username and/or password are missing, username is not an email, the passwords are not matching or if a user with that email already exists. All the errors are being caught using tests with appropriate error messages:

['Username must be a valid email']

The Password Field Is Required

The Username Field Is Required

The Username Field Is Required

A User With That Username Already Exists.

The Confirm Password Field Is Required

Passwords Are Not Matching.

The only potential issue is the error message that a username already exists in the database. It might be possible that a hacker could run a script off a collection of email addresses to check if any of them are registered in the app. In case they are, they might be potential entry points to the app. However, there are a number of ways in which someone could find email addresses and this probably would be the most painstaking. Also, if a user forgets the email with which they are registered and tries to register again, there should be a sensible error message that states that the email is already in the database. If the user cannot login with that email, they should reset the password. But for this action to be possible, the error message needs to be clear that the username exists. A vague error “Could not register user” may not give this information though it will also not give any indication that a user email exists in the database.

Next coming to the VerifyUserView which is after a user clicks on the verification link received by email. Here, the following conditions are possible – the verification is successful, the token has expired, the token is not genuine or if the user no longer exists. The last is if a user registers but does not click on the registration link for a long time, nor requests a new verification link. A cleanup script could delete all these inactivate accounts that have never been verified. The error responses are:

Token is invalid or expired

Token is invalid or expired

['User could not be activated']

The last error is in reality “User has been deleted”. However, there is no need to give out that much information. This might be the case where it is best not to disclose internal procedure to a user. The frontend need only display an error that activation failed and the user should try to ask for a new verification email or register again.

The next view is the ResendVerificationEmailView. This is when the user has not clicked the verification link in time. This is a get request with the user ID as a parameter. The valid case is when the user exists in the database. However, this is when it is clear that there is an issue with the send email utility method which is also used while registering a new user. The email method does not check if the user is already active. Though the verification link is something that will be used only in the beginning upon registration, it does have the capacity of becoming a nuisance if a number of random requests are made and all of them result in an attempt to send an email.

This only needs the send_verification_email method to be called inside the try block as it can throw an error that needs to be caught. And in the send_verification_method, only an additional check needs to be made if the user is activated and a ValidationError raised if that is the case.

In addition to this, there are a couple of issues with this view class. In the case of a success, it returns the email address corresponding to the user ID. This is a huge vulnerability as a get request with a random ID could return actual email addresses. In the request is a success, only a blank 200 response needs to be returned.

The second issue is that in case the user has been deleted or if the ID is simply random and does not correspond to a user, the error returned is “User matching query does not exist”. This is a core Django SQL error and should not be returned by the API as it is. In this case, a separate except block needs to be inserted that catches the error ObjectDoesNotExist from django.core.exceptions.

The next view class is the LoginUserView. This takes the username and password and either provides the JWT in the response or returns a 401 unauthorized.

The next view class is the ResetPasswordView. This also takes a user ID as parameter and sends a password reset link to the user email. A successful response is only a 200, while errors are if a user is not active which results in “No user to send email to” or if a user does not exist. The error for the user not existing is the same SQL error as before which needs that code refactor in the view class in the form of an extra except block.

Security analysis with a SAST tool

The first step in security analysis is a Static Application Security Testing (SAST) which is merely a review of the code to highlight segments that might be giving away too much information or might be known vulnerable code patterns. I use HCL AppScan which is an extension with VSCode and this highlights code vulnerabilities after saving the file. So far, none of the code in any of the files has any code which has highlighted vulnerabilities.

In the unit test files, there is no highlighting of security vulnerabilities even though there is clearly the hardcoding of credentials such as test user passwords etc. Which makes me wonder if AppScan excludes unit test files from the security scan. As a check, I chose a random file like models.py and inserted a dummy line with a hardcoded credential. This was immediately highlighted as a security vulnerability with the status ‘High’. However, the manner in which it picked up the vulnerability seems to indicate that it is not very robust. For example:

self.password = '123'

Was highlighted as a vulnerability, but:

self.set_password('123')

Was not highlighted though it should have been. This points to the fact that HCL AppScan may not be a very robust SAST tool for Python-Django code.

Another SAST tool that I have used in the past is semgrep though it was for JavaScript. Running a very basic scan using

semgrep scan --config=auto

Does return a few vulnerabilities though none of them are related to hardcoding the password but rather not validating the password which is also a vulnerability. At the moment, I do not want to implement a password validation as it would mean any password used during testing will fail as most passwords are plain text password that resemble common words. Here, only a basic test is being run and rules are not being finetuned and the test is completely offline. However, the fact that the vulnerabilities are more closely related to Django and web apps makes it more promising that AppScan.

Code refactoring

With the code gradually expanding, it is necessary to stop once in a while and find ways to make the code more DRY and also follow the Model-View-Controller approach to building a web app. So, in terms of DRY, a few lines of code were being continuously repeated in the view method and therefore, it seemed like something that could be inserted into a function. The first is returning a HTTP error response when the serializer throws an error:

def serializer_error_response(serializer):
    '''Return 400 HTTP error if serializer has errors'''
    error_list = [serializer.errors[e][0].title()
                  for e in serializer.errors]
    return Response(
        data=error_list[0],
        status=status.HTTP_400_BAD_REQUEST
    )

The next is when a HTTP error response is returned when a JWT is either invalid or expired:

def token_error_response(token):
    '''Return 400 HTTP error if JWT has errors'''
    error_list = [token.errors[e][0].title()
                  for e in token.errors]
    return Response(
        data=error_list[0],
        status=status.HTTP_400_BAD_REQUEST
    )

The next is when the change password request is being processed. Initially, I had placed the code in the view function itself. Which is probably not the best place as what we are doing is an update of the existing User model instance. Therefore, the code for updating the password is better placed in the create() method of the ChangePasswordSerializer:

def update(self, instance, validated_data):
    '''Update user password'''
    if validated_data.get('password') is None:
        raise serializers.ValidationError('New password missing')
    if not instance.is_active:
        raise serializers.ValidationError('User not found')
    instance.set_password(validated_data.get('password'))
    instance.save()
    return instance

With that the code in the view ChangePasswordView becomes much simpler:

user_obj = User.objects.get_user_by_token(verification_token)
user_form = ChangePasswordSerializer(
    user_obj,
    data=self.request.data
)
# Check for password match
if not user_form.is_valid():
    return serializer_error_response(user_form)
else:
    user_form.save()

Another refactoring that was done is the extraction of the user object from the JWT. This was also done in the view function, but this would be better placed in queryset in a manager for the User model.

class UserManager(BaseUserManager):
    '''Manager for the User model'''

    def get_user_by_token(self, token, *args, **kwargs):
        '''Return a user object from JWT'''
        user_data = RefreshToken(token)
        try:
            user_obj = self.get_queryset().filter(
                id=int(user_data['user_id'])
            )[0]
        except:
            raise ValidationError('User not found')
        return user_obj

    def activate_user_by_token(self, token, *args, **kwargs):
        '''Activate a user from JWT'''
        try:
            user_obj = self.get_user_by_token(token)
            user_obj.is_active = True
            user_obj.save()
            return user_obj
        except:
            raise ValidationError(
            	'User could not be activated'
            )

There is a base query get_user_by_token which attempts to extract a user object from the JWT. This could fail for many reasons – the token is not genuine and does not result in a sensible user_data object or does not have the user_id claim. In this case, a User not found validation error will be generated. The other problem is that the user with that ID may not be found, which again will produce the same error.

The other query activate_user_by_token references the get_user_by_token query to fetch a user object and sets the is_active attribute to True. These two queries also make the view function cleaner as handling the database can now be moved into the manager which deals with the User table.

Completing the basic user endpoint collection

Took a week’s break from this project and finally got back to a couple of days back. To continue with the API, the next that is needed is a login end-point, a reset password end-point and a change password endpoint.

The LoginUserView is quite easy – all that is needed is to use the authenticate method from django.contrib.auth and pass it the username and password from the request:

class LoginUserView(APIView):
    '''Login user and return token'''
    serializer_class = UserSerializer

    def post(self, *args, **kwargs):
        user_obj = authenticate(
            username=self.request.data.get('username', None),
            password=self.request.data.get('password', None)
        )
        if user_obj is not None:
            user_token = RefreshToken.for_user(user_obj)
            return Response(
                data=str(user_token),
                status=status.HTTP_200_OK
            )
        else:
            return Response(
                data='Invalid username/password',
                status=status.HTTP_401_UNAUTHORIZED
            )

The frontend will then use the JWT in all subsequent requests.

The ResetPasswordView is simply a get request with the user ID that will send an email to the user with a link through which he can change his password. Here I do see some security challenges which will be addressed later.

class ResetPasswordView(APIView):
    '''Send password reset link to user email'''

    def get(self, *args, **kwargs):
        user_id = self.kwargs['user_id']
        try:
            user_obj = User.objects.get(id=user_id)
            send_password_reset_email(user_obj)
        except Exception as e:
            return Response(
                data=str(e),
                status=status.HTTP_400_BAD_REQUEST
            )
        return Response(
            status=status.HTTP_200_OK
        )

The password reset email will merely have a link to /api/user/change-password with a new token for the user. Similar to the verification link, this token will expire in 15 minutes.

The next is the ChangePasswordView. In this case, we are performing an update but only with respect to the password. The username cannot change. For this reason, using the UserSerializer or the RegisterUserSerializer is not possible as these have the username as a field. For this reason, a ChangePasswordSerializer has been defined:

class ChangePasswordSerializer(RegisterUserSerializer):
    '''Serializer for resetting the password of a user'''

    def update(self, instance, validated_data):
        '''Update user password'''
        if validated_data.get('password') is None:
            raise serializers.ValidationError('New password missing')
        if not instance.is_active:
            raise serializers.ValidationError('User not found')
        instance.set_password(validated_data.get('password'))
        instance.save()
        return instance

    class Meta(RegisterUserSerializer.Meta):
        model = User
        fields = ['password', 'confirm_password']

The ChangePasswordSerializer can inherit the RegisterUserSerializer as that performs the validation that the password and confirm_password fields are the same. The only difference is that the username field is removed from the fields attribute in the Meta class.

class ChangePasswordView(APIView):
    '''Change a user password'''

    serializer_class = ChangePasswordSerializer

    def post(self, *args, **kwargs):
        verification_token = self.kwargs['token']
        if not verification_token:
            return Response(
                data='Missing token',
                status=status.HTTP_400_BAD_REQUEST
            )
        token_data = TokenRefreshSerializer(
            data={'refresh': verification_token}
        )
        try:
            # Check for expired token
            if not token_data.is_valid():
                return token_error_response(token_data)
            else:
                user_obj = User.objects.get_user_by_token(
                     verification_token
                )
                user_form = ChangePasswordSerializer(
                    user_obj,
                    data=self.request.data
                )
                # Check for password match
                if not user_form.is_valid():
                    return serializer_error_response(user_form)
                else:
                    user_form.save()
        except Exception as e:
            return Response(
                data=str(e),
                status=status.HTTP_400_BAD_REQUEST
            )
        return Response(status=status.HTTP_200_OK)

These endpoints have also been tested for successful as well as failed responses. The only other endpoint that remains with respect to basic user account handling is a logout functionality. Because JWT is being used, this is not very easy as JWTs are stateless with no entry being written into the database, and therefore no token can be deleted from the database. The other option is to blacklist a JWT when a user requests a log out and if any more requests are made with the JWT, these requests will be declined as long as the JWT has not expired. However, it might be easier to just implement a logout functionality only on the frontend. This still implies someone can make API requests using a JWT independent of a browser or mobile app. This might be worth investigating later, as from a security perspective, once a user logs out, it is advisable to clean out the session and not allow any more activity on the user account.

Resend verification link email API endpoint

Now that the endpoint for handling clicks of the verification link is done, the next should be an endpoint that resends this verification link. This is necessary in case the user did not click on the verification link before it expired. The code for this is fairly simple, but I stumbled onto another issue which will also be a part of this blog post.

class ResendVerificationEmailView(APIView):
    '''Resending verification email to user'''

    def get(self, *args, **kwargs):
        user_id = self.kwargs['user_id']
        try:
            user_obj = User.objects.get(id=user_id)
        except Exception as e:
            return Response(
                data=str(e),
                status=status.HTTP_400_BAD_REQUEST
            )
        send_verification_link_email(user_obj)
        user_data = UserSerializer(user_obj)
        return Response(
            data=user_data.data,
            status=status.HTTP_200_OK
        )

The test for this endpoint is quite simple:

def test_resend_verification_endpoint(mock_send_email, test_user):
    '''Testing endpoint for resending verification email'''
    client = APIClient()

    api_response = client.get(
        '/user/resend-verification/{user_id}'.format(user_id=test_user.id),
        format='json'
    )
    assert api_response.status_code == 200

    old_user_id = test_user.id
    test_user.delete()
    api_response = client.get(
        '/user/resend-verification/{user_id}'.format(user_id=old_user_id),
        format='json'
    )
    assert api_response.status_code == 400

The test creates a user and makes a GET request with the user ID. If the user does not exist, it should return a 400 response, or else it should return a 200 response. As before with testing the user registration, I do not want to be sending an email or even trying to send one. Except that the mocking of the send email method is done in a fixture mock_send_email which appears in the test function argument.

import pytest

@pytest.fixture
def mock_send_email(monkeypatch):
    monkeypatch.setattr(
        'user_auth.views.send_verification_link_email',
        lambda user: print(
            'Sending email to {username}'.format(username=user.username)
        ),
        raising=True
    )

The change made is in the method being mocked – user_auth.views.send_verification_link_email. To begin with, the entire path is being used to make it explicit which method is being mocked. However, the method send_verification_link_email is not being mocked in utils.py module where it is defined but rather in views.py module where it is being used. Here is the difference with respect to mocking in JavaScript. In JavaScript, one would create a spy that would replace the original object with a spy. In Python, one does not replace the original object but rather merely replaces the reference of the object where it is being used. So the original email method in utils.py is not being mocked, but rather the usage of this method in views.py is being mocked.

For convenience this mock fixture is placed in a separate fixtures.py file as it can also be used in the test for user registration and so keep the code DRY.

Testing token verification endpoint

To test the API endpoint that verifies the verification link click, I needed to do some research on Pytest fixtures. Though I could have used fixtures in the previous tests, now there is a good use case as in order for the verification link to work, there must be a user already in the database and for this user we need to create a verification token. Both of these tasks would be best handled by a fixture to keep the test DRY.

For the creation of a user:

@pytest.fixture
def test_user():
    '''Create sample user for test'''
    sample_user = User.objects.create(username='someuser@somedomain.com')
    sample_user.set_password('somepassword')
    sample_user.save()
    return sample_user

All the fixture needs is the @pytest.fixture decorator and the return value of the function can be used directly as an object in the test function or another fixture. The fixture for creating the verification token will however be a little more complicated as the expiration time of the token needs to be dynamic. For this I use the “Factories as fixtures” feature with fixtures as described in the documentation:

@pytest.fixture
def verification_token(test_user):
    '''Creating tokens with JWT'''

    def _create_token(exp_time):
        '''Token with variable expiry time'''
        verification_token = RefreshToken.for_user(test_user)
        verification_token.set_exp(
            from_time=verification_token.current_time,
            lifetime=timedelta(seconds=exp_time)
        )
        return verification_token

    return _create_token

This fixture uses the test_user fixture. Instead of returning a token, the fixture returns another method which accepts the expiry time as an argument and returns the verification token. This can be used in the test function as:

def test_user_verification_endpoint(verification_token, test_user):
    '''Testing the verify-email endpoint'''
    client = APIClient()

    # Token with 60sec validity
    test_token1 = verification_token(60)

    api_response = client.get(
        '/user/verify-user/{token}'.format(
            token=test_token1
        ),
        format='json'
    )
    assert api_response.status_code == 200
    assert test_user.is_active == True

So here invoking the fixture name verification_token produces a function as the fixture does not return a value but rather a function. This function can be passed the expiration time. The above test verifies that the verification link works - is linked to a user in the database and also it results in the user becoming active. A few failing API requests can also be tested:

# Token with 1sec validity - expired token test
test_token2 = verification_token(1)
# Sleep for 2sec
time.sleep(2)

api_response = client.get(
    '/user/verify-user/{token}'.format(
        token=test_token2
    ),
    format='json'
)
print(api_response.data)
assert api_response.status_code == 400

# Tampered token test
test_token3 = str(verification_token(60))
test_token3 = test_token3[:-1]

api_response = client.get(
    '/user/verify-user/{token}'.format(
        token=test_token3
    ),
    format='json'
)
print(api_response.data)
assert api_response.status_code == 400

# Deleted user test
test_token4 = verification_token(60)
test_user.delete()

api_response = client.get(
    '/user/verify-user/{token}'.format(
        token=test_token4
    ),
    format='json'
)
print(api_response.data)
assert api_response.status_code == 400

I printed out the error messages and the last message is interesting:

Token is invalid or expired
Token is invalid or expired
User matching query does not exist.

This is because I am trying to fetch the user from the database with the user_id found in the JWT payload.

Creating verification link and handling verification requests

With email sending configured, the next step is to send a verification link to a newly registered user. Upon clicking this verification link, the user account should become active. This can be done in the send_verification_link_email util method. To create tokens, I am using the Simple JWT package that is recommended with Django Rest Framework.

from rest_framework_simplejwt.tokens import RefreshToken

verification_token = RefreshToken.for_user(user)
verification_token.set_exp(
    from_time=verification_token.current_time,
    lifetime=timedelta(minutes=settings.EMAIL_VERIFICATION_TIMELIMIT)
)

To create the verification link, rather than format the entire URL as a string, I use a combination of Django settings and URL reverse lookups. To begin with, the method used to include the URL end points for the user_auth app was wrong. In the main urls.py file in server directory, I need to define a namespace for the user_auth app.

urlpatterns = [
    path('admin/', admin.site.urls),
    path('user/', include('user_auth.urls', namespace='user_auth')),
]

And in the urls.py in the user_auth module, I need to define an app_name:

app_name = 'user_auth'
urlpatterns = [
    path('register-user', RegisterUserView.as_view(), name='register-user'),
    path('verify-user/', VerifyUserView.as_view(), name='verify-user'),
]

It is possible to extract the domain name along with the protocol (http or https) from the request object. But rather than pass the request object to a utility function, the base domain name along with the protocol can be a setting parameter which will change with the environment. So, in the settings.py file:

# Base URL
BASE_URL = KEYS.BASE_URL

And in the enviromment, this BASE_URL can be defined as http://localhost:8000 which is the development server. Once this is done, using string formatting and URL reverse lookups, the verification link can be inserted into email message.

message_body = (
    "Hello,\n"
    "Thank you for registering with Online Edu!\n"
    "\n"
    "You are not yet ready to use your account.
    Before you can login to the website, 
    please verify your email by clicking on this link:\n"
    "{base_url}{token_url} \n"
    "\n"
    "Please click on this link within 15 minutes of receiving this email.\n"
    "\n"
    "Thank you,\n"
    "Online Edu"
).format(
    base_url=settings.BASE_URL,
    token_url=reverse(
        'user_auth:verify-user',
        args=[verification_token]
    )
)

With this, the amount of hardcoding is the least and with changes in deployment or for that matter in the URL end-points, no changes will need to be made in the utility method that sends the email.

To handle the verification, we need another view class. From the Simple JWT documentation, there are ready made views that extract the token and verify it. However, in my case, I need to verify the token and if verified, the new user account needs to be set to active. Sadly, the Simple JWT documentation is very scant and to figure out all that can be done by this package, it is necessary to browse through the Github code. For now, this view class will do as clicking the link is a GET request with the token as a URL parameter rather than embedded in the header which would be the case with more API requests. Therefore, a lot of the code is custom, though it might be possible to replace it later with boilerplate code.

class VerifyUserView(APIView):
    '''Checks token received on clicking verification link'''

    def get(self, *args, **kwargs):
        verification_token = self.kwargs['token']
        if not verification_token:
            return Response(
                data='Missing token',
                status=status.HTTP_400_BAD_REQUEST
            )
        token_data = TokenRefreshSerializer(
            data={'refresh': verification_token}
        )
        try:
            if not token_data.is_valid():
                error_list = [token_data.errors[e][0].title()
                              for e in token_data.errors]
                return Response(
                    data=error_list[0],
                    status=status.HTTP_400_BAD_REQUEST
                )
            else:
                # Set the user to active
                user_data = RefreshToken(verification_token)
                user_obj = User.objects.get(id=int(user_data['user_id']))
                user_obj.is_active = True
                user_obj.save()
        except Exception as e:
            return Response(
                data=str(e),
                status=status.HTTP_400_BAD_REQUEST
            )
        return Response(status=status.HTTP_200_OK)

If no token is passed to the URL end point or if the token is invalid or has been tampered with, the view will return a 400 bad request response. There is a TokenRefreshSerializer serializer in Simple JWT which takes the token as data. This then has a validation method which checks for expiry and also things like blacklisting of tokens. Though this is a serializer, it does not seem possible to extract the .data attribute and then check the payload for the user information. To be able to do this, I import the RefreshToken model from SimpleJWT and pass the verification token to it. This allows to extract the user_id from the payload. This is necessary to set the is_active to True.

This is the first task that has resulted in me thinking about JWTs and how they can be used. Specifically, how to handle an expired token and what can be done with it. This is to be handled on the frontend as a user who clicks on the verification link after it expires will need a refresh options.

Configuring email sending through Django

Now that the register user API end point is ready and tested, the next step is to send the verification email where the user will click on a link to verify his or her email. Before going into the nature of the verification link, the first step is to configure email sending through Django.

Django comes with the django.core.mail package with the send_mail method. So no additional packages are needed. This needs a few additional configurations in the email client that the admin (in this case me) will be using and also in the Django settings.py file. I am using my Gmail client. For this, I needed to go to “Manage your Google Account” and in there to go to the “Security” section. Here, one needs to turn on “2-step Verification”. In reality, the be able to use the client does not have anything to do with 2-step verification, but the option to create passwords for third party apps lies within this option. Way at the bottom of this 2-step verification section, there is “App passwords”. Here one needs to create an app and it will provide a password. This password is what will be used by Django for sending emails through the client.

The additional settings in settings.py file are:

# Email settings
EMAIL_BACKEND = 'django.core.mail.backends.smtp.EmailBackend'
EMAIL_HOST = KEYS.EMAIL_HOST
EMAIL_HOST_USER = KEYS.EMAIL_USERNAME
EMAIL_HOST_PASSWORD = KEYS.EMAIL_PASSWORD
EMAIL_PORT = KEYS.EMAIL_PORT
EMAIL_USE_TLS = True
DEFAULT_FROM_EMAIL = KEYS.EMAIL_FROM_USERNAME

Since the settings are sensitive or variable, it is best to import them from the environment which for now is merely an env.py file until this app gets deployed. The keys are:

# Email settings
EMAIL_HOST = 'email-host'   # for gmail - 'smtp.gmail.com'
EMAIL_USERNAME = 'username'
EMAIL_FROM_USERNAME = 'from-username'   # can be same as username
EMAIL_PASSWORD = 'password'   # set app-password in Security of Google account
EMAIL_PORT = 123              # 587 for SMTP if using google

These entries are from env_example.py file which describes the keys that need to be added to env.py file. The examples are specified for Gmail though other clients can also offer similar features.

With this done, all that needs to be done is call the send_mail method. Instead of calling it from RegisterUserView class in views.py file, it would be better to place this code in a utility method defined in a file utils.py file in user_auth module. This would only make it easier to test and mock the method in tests.

def send_verification_link_email(username):
    '''Send an email to newly registered used with verification link'''

    message_body = (
        "Hello,\n"
        "Thank you for registering with Online Edu!\n"
        "\n"
        "You are not yet ready to use your account.
        Before you can login to the website,
        please verify your email by clicking on this link:\n"
        "http://www.google.com \n"
        "\n"
        "Please click on this link within 15 minutes of receiving this email.\n"
        "\n"
        "Thank you,\n"
        "Online Edu"
    )

    send_mail(
        subject='Verification link',
        message=message_body,
        from_email=settings.DEFAULT_FROM_EMAIL,
        recipient_list=[username]
    )

As of now the verification link is a dummy link that has been inserted. The next step will be to use JSON Web Tokens to create a token and then a link with that token.

But before that, the tests need to be updated as sending email should not happen during tests and this method should be mocked.

Confirming password while registering

To add the confirm_password field when registering a new user, the simplest way seems to be to define another serializer RegisterUserSerializer that inherits the UserSerializer:

class RegisterUserSerializer(UserSerializer):
    '''Serializer for registering a new user'''
    confirm_password = serializers.CharField(
        style={'input_type': 'password'},
        write_only=True
    )

    def validate(self, data):
        '''Validate that password and confirm_password are the same'''
        if not data['password'] == data['confirm_password']:
            raise serializers.ValidationError('Passwords are not matching.')
        return data

    class Meta(UserSerializer.Meta):
        model = User
        fields = ['username', 'password', 'confirm_password', 'is_active']

A new field confirm_password has been defined along with a validate method that performs object level checking on the serializer to ensure that the password and confirm_password field are identical. The view method will now use request.data to create a RegisterUserSerializer instance instead of UserSerializer. The only problem is that the error for passwords not matching is not very nice – “non_field_errors – Passwords not matching”. This is because errors found on the object using the validate method are sent out in the dictionary with non_field_errors key. This means that the error messages will need to be customized rather than processed in the view method.

class Meta:
    model = User
    fields = ['username', 'password', 'is_active']
    read_only_fields = ['is_active', ]
    extra_kwargs = {
        'username': {
            'error_messages':
            {
                'blank': 'The username field is required',
                'required': 'The username field is required'
            }
        }
    }

I included both blank and required error messages for the browser form as well as an API request sent from Postman. The password fields can have similar error messages in their serializer field definitions.

password = serializers.CharField(
    style={'input_type': 'password'},
    write_only=True,
    error_messages={
        'blank': 'The password field is required',
        'required': 'The password field is required'
    }
)

confirm_password = serializers.CharField(
    style={'input_type': 'password'},
    write_only=True,
    error_messages={
        'blank': 'The confirm password field is required',
        'required': 'The confirm password field is required'
    }
)

The tests for the API endpoint can be updated:

def test_register_new_user():
    '''Test API to register new user'''
    client = APIClient()

    # Should result in a user created in db
    api_response = client.post(
        '/user/register-user',
        {
            'username': 'someuser@domain.com',
            'password': 'somepass',
            'confirm_password': 'somepass'
        },
        format='json'
    )
    assert api_response.status_code == 201
    assert api_response.data['username'] == 'someuser@domain.com'
    assert hasattr(api_response.data, 'password') == False
    assert api_response.data['is_active'] == False

    users_in_db = User.objects.all().count()
    assert users_in_db == 1

    # Should fail model validation
    api_response = client.post(
        '/user/register-user',
        {
            'username': 'someuser',
            'password': 'somepass'
        },
        format='json'
    )
    assert api_response.status_code == 400

    # Should fail because of missing password field
    api_response = client.post(
        '/user/register-user',
        {
            'username': 'someuser@domain.com',
        },
        format='json'
    )
    assert api_response.status_code == 400

    # Should fail because of missing username field
    api_response = client.post(
        '/user/register-user',
        {
            'password': 'somepass',
        },
        format='json'
    )
    assert api_response.status_code == 400

    # Should fail become of missing username and password
    api_response = client.post(
        '/user/register-user',
        format='json'
    )
    assert api_response.status_code == 400

    # Should fail because of missing confirm_password field
    api_response = client.post(
        '/user/register-user',
        {
            'username': 'someuser@domain.com',
            'password': 'somepass',
        },
        format='json'
    )
    assert api_response.status_code == 400

    # Should fail because the passwords do not match
    api_response = client.post(
        '/user/register-user',
        {
            'username': 'someuser1@domain.com',
            'password': 'somepass',
            'confirm_password': 'somepass1'
        },
        format='json'
    )
    assert api_response.status_code == 400

    users_in_db = User.objects.all().count()
    assert users_in_db == 1

Testing the register user API endpoint

Writing tests for the API call threw up a few basic errors that I had overlooked along with the fact that I had forgotten to add the confirm password field. The confirm password field will come next, but first the tests and the changes in the code to make the tests pass.

To begin with the most basic test for the API:

from rest_framework.test import APIClient

def test_register_new_user():
    '''Test API to register new user'''
    client = APIClient()

    # Should result in a user created in db
    api_response = client.post(
        '/user/register-user',
        {
            'username': 'someuser@domain.com',
            'password': 'somepass'
        },
        format='json'
    )
    assert api_response.status_code == 201
    assert api_response.data['username'] == 'someuser@domain.com'
    assert hasattr(api_response.data, 'password') == False
    assert api_response.data['is_active'] == False

    users_in_db = User.objects.all().count()
    assert users_in_db == 1

However, even something as this fails. Examining the api_response.data shows Bad request. The reason is that I have been using request.POST for populating the UserSerializer. Since I am using Request from DRF, I do not need to use request.POST as that still contains the encodings with a submitted form. Instead I can use request.data which contains a dictionary with the fields. This way, the API extracts the username and password whether a form was filled with the link accessed in a browser or the data submitted through Postman or in this case the test.

The tests for the API can be expanded to a few other cases:

# Should fail model validation
api_response = client.post(
    '/user/register-user',
    {
        'username': 'someuser',
        'password': 'somepass'
    },
    format='json'
)
assert api_response.status_code == 400

# Should fail because of missing password field
api_response = client.post(
    '/user/register-user',
    {
        'username': 'someuser1@domain.com',
    },
    format='json'
)
assert api_response.status_code == 400

# Should fail because of missing username field
api_response = client.post(
    '/user/register-user',
    {
        'password': 'somepass',
    },
    format='json'
)
assert api_response.status_code == 400

# Should fail become of missing username and password
api_response = client.post(
    '/user/register-user',
    format='json'
)
assert api_response.status_code == 400

# Should fail because user has been created above
api_response = client.post(
    '/user/register-user',
    {
        'username': 'someuser@domain.com',
        'password': 'somepass'
    },
    format='json'
)
assert api_response.status_code == 400

users_in_db = User.objects.all().count()
assert users_in_db == 1

In the case of certain tests such as when an API request is submitted without a password field, an error is returned that this field is required. However, the error message merely is “This field is required” as the error returned is from the UserSerializer validator for the password field. To make it clear that the field is missing, the error response returned in the view has to be changed to:

error_list = [
    '{error_field}-{error_text}'.format(
        error_field=e,
        error_text=user.errors[e][0].title()
    ) for e in user.errors
]

This formatting of the error string will make it clear with the field before the error message. Something that is interesting in the errors in the API responses is the error when the test attempts to create a duplicate user. The error is generated from the User model that is in-built with Django. However, examining the error message gives “username-A User With That Username Already Exists.” This shows that the error is generated by the serializer rather than the model. How does the error from the model get raised to the level of the Serializer that is created from the model? This is something that is enforced at the level of the database as the username is a unique field.

The next change that needs to be made is that a confirm_password field needs to be added when registering a user.

Registering a new user

With the custom User model created and the additional code to enforce the username to be a valid email, the next step is to allow a new user to register with an API call. For this, I get started with Django Rest Framework. After the usual installation and inclusion of ‘rest_framework’ in INSTALLED_APPS in settings.py file, DRF is ready to be used.

The first step is to create a UserSerializer class based on the custom User model:

class UserSerializer(serializers.ModelSerializer):
    '''Serializer for User model'''
    password = serializers.CharField(
        style={'input_type': 'password'},
        write_only=True
    )

    # def validate_username(self, value):
    #     '''Check that username is a valid email'''
    #     try:
    #         validate_email(value)
    #     except:
    #         raise serializers.ValidationError('Username must be a valid email')
    #     else:
    #         return value

    def create(self, validated_data):
        '''Create new user in db'''
        new_user = User(
            username=validated_data['username']
        )
        new_user.set_password(validated_data['password'])
        # User should be inactive until email is validated
        new_user.is_active = False
        new_user.save()
        return new_user

    class Meta:
        model = User
        fields = ['username', 'password', 'is_active']
        read_only_fields = ['is_active', ]
        # extra_args = {
        #     'password': {
        #         'write_only': True
        #     }
        # }

Though documentation on Serializers and ModelSerializers is fairly extensive, to be able to code it needs a bit of googling and experimenting. The simplest implementation is merely to define:

class UserSerializer(serializers.ModelSerializer):
    '''Serializer for User model'''
    class Meta:
        model = User
        fields = ['username', 'password', 'is_active']

Next is to use the CreateAPIView with DRF to create a new model instance – in this case to register a new user.

class RegisterUserView(CreateAPIView):
    '''Register a new user'''
    serializer_class = UserSerializer

To be able to access this, we need to register a URL for the same. Since several urls will be related to the User model, a separate urls.py file can be created in the user_auth app directory. This will have the url:

urlpatterns = [
    path('register-user', RegisterUserView.as_view(), name='register-user'),
]

The urls.py file in user_auth app directory can be included in the main urls.py file as:

from user_auth.urls import urlpatterns as user_urlpatterns

urlpatterns = [
    path('admin/', admin.site.urls),
    path('user/', include(user_urlpatterns)),
]

Now this URL can be accessed in the web-browser with the development server on the link user/register-user/ and this will give a form for the user. This form will have three fields – username, password and is_active. To begin with, the is_active field is not for the user to enter, but to be handled by the backend. The intention is to create a new user upon registration, but to set the is_active to False. When an email is set to the user’s email, the user will confirm the registration and only then will the is_active be set to True. Of course, the email integration will come later.

But first, this means that the is_active field should not appear in the default form. For that, the read_only_fields attribute need to be specified in class Meta. The next change that needs to be made is that the password is something that should only be written and should not be returned in an API response. For that, we can use the attribute extra_args and specify attributes for any of the fields. This is done with:

extra_args = {
    'password': {
        'write_only': True
    }
}

The problem however is that the password form field in the browser does not hide the password and it appears in plain text. It might be possible to add more parameters to the password field in the extra_args. But, in such a case, it might be best to redefine the password field as a serializer field:

password = serializers.CharField(
    style={'input_type': 'password'},
    write_only=True
)

Here, the style has been defined with the input_type being password. In such a case of redefinition, the parameters specified in extra_args gets overwritten and so it is best to define all parameters and attributes in this redefinition itself.

With this, the form for registering a user looks OK – the password is hidden and is_active is not shown to the user. To be able to handle the form submission, the CreateAPIView provides a post method handler:

def post(self, *args, **kwargs):
    '''Create new user from API request'''
    user = UserSerializer(data=self.request.POST)
    if user.is_valid():
        try:
            user.save()
        except Exception as e:
            return Response(
                data=e,
                status=status.HTTP_400_BAD_REQUEST
            )
    else:
        error_list = [user.errors[e][0].title() for e in user.errors]
        return Response(
            data=error_list[0],
            status=status.HTTP_400_BAD_REQUEST
        )
    return Response(user.data, status=status.HTTP_201_CREATED)

The advantage of specifying the serializer_class is that the data in the form can be directly dumped into the serializer from the POST data. One can check whether the serializer data is valid and only then create the model. But here there is the catch. The validation that has been created at the User model does not directly translate to the UserSerializer class even though it is based of the User model. I suppose one would have to redefine the fields of the UserSerializer with validation parameters. But it would defeat the purpose of redefining all fields of a User. So this means that checking for validity of UserSerializer data will not provide errors such as the username not being a valid email.

To be able to check for validity of UserSerializer data, it is also possible to define a validate_username method:

def validate_username(self, value):
    '''Check that username is a valid email'''
    try:
        validate_email(value)
    except:
        raise serializers.ValidationError('Username must be a valid email')
    else:
        return value

Defining such a method seems worse that redefining the serializer class fields as now this is repeating code. So, I have commented it out for now as the model validation should be good enough. This implies that at this moment the is_valid() check is useless as the serializer does not have any validation. I have kept it for now in case I add other validation in UserSerializer.

To be able to create a new user while also ensuring that the password is hashed and not stored directly as raw password, the create method of UserSerializer needs to be overridden:

def create(self, validated_data):
    '''Create new user in db'''
    new_user = User(
        username=validated_data['username']
    )
    new_user.set_password(validated_data['password'])
    # User should be inactive until email is validated
    new_user.is_active = False
    new_user.save()
    return new_user

Here, the checking that the username has to be valid email is performed and it will raise a ValidationError if that fails. So, the create method either returns the successfully created user or raises a ValidationError. The view then returns a 201 response with the new user or the validation error as error message with 400 response.

This works well with manual testing. Next, to clean up code and write tests.