Design and Implementation of Open Source Online Evaluation System Based on Cloud Platform

: In order to provide a good practice platform for the program design contestants and algorithm enthusiasts, this paper designs and implements a programming online evaluation system based on cloud platform, which is a web system that can return the test results of the program source codes submitted by users in real time. It realizes the automatic evaluation of the program design training questions. The system is implemented by the way of front-end and back-end separation and modular programming. The front-end of the web is implemented by Vue framework, the back-end is implemented by Django framework. The judgment core is written in C language for the sake of improving the efficiency of the system. Sandbox technology is used to ensure the security of the judgment system. Experimental results show that the system can meet the need of daily teaching and competition.


Introduction
OJ (online judge system) is an online system widely used in teaching and programming competitions. The system can accept the source code submitted by the user online, compile and run it online, and determine the correctness of user code using the test data provided in advance [1].
Online evaluation system at home and abroad is generally developed based on Linux platform [2], which brings convenience to development, but also brings some obstacles to the use of the system. Although the Linux community in China has developed rapidly in recent years, there are not many teachers and students who are really proficient in using the Linux operating system platform in domestic universities. Even if the source code of the system program is obtained, it is difficult to actually deploy it successfully.
There are many online judge systems that can be accessed on the Internet, while the number of open source online evaluation systems is relatively limited. In the open source system, many of them are experimental systems developed for the purpose of graduation project, so they are lack of practical value. Among the well-known online evaluation systems in China, Zhejiang University online judge system (ZOJ) and Beijing Jiaotong University online judge system (BOJ) have made open-source attempts. However, it is not enough to open source code merely, and documentation, community and effective technical support are necessary. Due to the lack of documentation and slow code update, the online evaluation system developed by the two well-known universities is rarely used by other schools. Moreover, ZOJ uses J2EE framework and C/S structure based on socket. Its system complexity and hardware system requirements are relatively high, so it is difficult to deploy in the open cloud platform. However, other small-scale open source evaluation systems often lack scalability, and there are fundamental defects in the use of distributed applications or cloud platforms.
HUSTOJ is an online judge system developed by Wang Liangjing, a graduate of Huazhong University of science and technology, and open-source by GPL protocol at the end of 2008. HUSTOJ developed on lamp platform, and its early version has been running stably on the server of Huazhong University of science and technology up to now. In the early days, HUSTOJ, like ZOJ and BOJ in terms of open source, lacked due documentation and technical support. Open cloud platform is a new distributed computing application platform proposed by internet giants in recent years. First of all, a huge infrastructure is designed and built by an Internet company with strong capital and technical strength, including a distributed service network composed of thousands of computing nodes distributed around the world. Then, it is published by the system builder to the individuals, companies and organizations who need to build web services in a transparent way. The open cloud platform provides an economic and reliable way to build high-performance web services for individuals and small and medium-sized organizations. This is in line with the requirements of universities and other scientific research institutions for building high-performance online judge system.
At present, the most popular cloud platforms in foreign countries are Amazon and Google's cloud platforms [3][4][5]. The better open cloud platform service in China is SAE (SINA App Engine) provided by SINA, a well-known listed company. SINA cloud platform provides services such as PHP5, MySQL, Memcache, Crontab, which are the most popular web development services in China. Compared with foreign platforms, the biggest advantage of SINA cloud platform is that its system is almost fully compatible with the common lamp (Linux, Apache, MySQL, PHP) platform. Many PHP applications can run smoothly in SINA cloud without modification or a few modifications. This brings great convenience for online evaluation system to migrate from non cloud platform to cloud platform. SINA cloud platform completely isolates the underlying services and applications in WebService pools through service router, realizes the seamless deployment of system resources in operation, and can dynamically add and delete nodes participating in services without affecting the application operation. The mature reverse proxy mechanism can greatly ensure the efficiency of system services, improve the response speed and effectively reduce the running load of basic services.
The open-source online evaluation system based on cloud platform can meet the application requirements from daily teaching to large-scale online competition. It can not only provide better use experience, but also solve the difficulties of building online evaluation system from four aspects: system implementation, system construction, operation and maintenance platform and topic acquisition. Compared with the common distributed web service platform, it has more expansibility and lower cost, which provides a new way for the improvement of online evaluation system.

Structure
OJ system is composed of Web front-end interface, back-end function module, MySQL database and test server. The web front-end provides an operation interface for administrators and users. Users can browse topics online, submit codes, view submission results, rank the number of problems solved, and report error information, search and other functions. Administrators have more functions than ordinary users, such as uploading questions and data. The database stores the questions and relevant data uploaded by the administrator, as well as the codes submitted by the users and the results of problem determination, and separates the questions and codes to reduce the problem of large data transmission volume of the server.
In fact, the most important part of the system is the design of the core and the security of the system, which are also the most difficult to design. The problem determination kernel needs to give correct evaluation results for each code submitted by the user, which must deal with the problem that the answers to special problems are not unique and the system has high concurrency; in addition, the security problem must be solved by each system. For this problem, we use the sandbox system to build a safe and independent operating environment for the system to avoid malicious code and network attacks. The system overall structure is shown in Fig. 1.

Client MySQL
Judge Insert user commit code to database Read the status from the database and display Figure 1: System overall structure diagram

Proposed Method
The core of this system is three parts: docker, Vue and Django, which will be introduced one by one.

Docker
Docker is an open source application container engine, which can be used by developers to set up a running environment and customize sandbox content. It is an architecture mode based on C/S, which has four parts: docker client, docker daemon, docker image, docker container. Docker container is created by docker image and managed by remote API. Docker client converts user's operations into executable commands and passes them to docker daemon for processing. Because docker uses virtualization to load docker containers according to user requirements, docker has a great advantage in building an isolated standardized operating environment: fast startup and small resource consumption.
The system only needs to install docker version 1.9.1 or above, and then write docker file to build the basic image. Through the instructions of docker run, docker build and so on, it can realize one click deployment, which is simple and convenient.

VUE
Vue is a lightweight progressive technology stack based on JavaScript [6]. It uses API to build data driver and realize bi-directional data binding. Its lightweight is reflected in the design of bottom-up incremental development. Vue is concentrated in the MVVM mode view layer, which is easier to use than react and angular, and Vue is module friendly: NPM, Bower or Duo installation does not force the code to follow the rules of angular, which is applicable to more scenarios.
NPM is used to install the system, and the official command-line tools provided by vue.js are used to quickly build applications.
This system realizes the data communication between the system interface and the database through Vue: the front end stores the code submitted by the user into the database, passes the topic ID to the problem determination kernel, and reads out the status (AC/Compile/Runtime/Wrong) from the database and displays it on the interface.

Django
Django is a MVC pattern framework based on Python, but it is different from other MVC pattern frameworks. Django pays more attention to model, template and views, that is, MTV pattern. The convenience of template is to use regular expression, match URL patterns in configuration information one by one, and replace data by returning HTTP response object.
The system uses MySQL database to store data, establishes the connection between database and user by setting the database item in settings.py file, and establishes the corresponding table structure by establishing Django model.

Design of the Core of Judging Question
The following is an introduction to the types of topics. How to optimize the evaluation mode, improve the evaluation speed and improve the user experience is the reasonable design of the evaluation mechanism.
First, there are two types of questions. The first type is that the running result of the program is almost the same as the uploaded output file. At this time, the program is considered to be correct and accept. Otherwise, there are other errors, such as presentation error, wrong answer and so on. The second is that for the same problem, players may use different algorithms and programs to solve it. At this time, for some special problems, the expected results may not be unique, but they are all in line with the meaning of the problem and meet the requirements. At this time, special judge is required. Generally, the answers to such problems are not unique, and then the accuracy needs to be controlled.
In this way, firstly, OJ system is roughly divided into performance layer, business layer, judgment kernel, persistence layer, and there is a series of data transmission between each layer [7][8][9][10]. First, judge that the kernel gets the task submitted by the user in the business layer, add its task to the queue, and then the evaluation machine takes the first job of the team for evaluation, and sends the evaluation result to the persistence layer, at the same time, it reverses the evaluation result It is fed to the service layer and finally transmitted to the performance layer, so that an evaluation is completed. The specific evaluation process is. First, the parent process selects the first job in the queue according to the job queue, then queries the database to set the memory space and execution time of the program according to the requirements of the current topic and the user programming language, creates a child process, and then performs the compilation, writes the compiled information to the file, if the compilation fails, the parent process directly returns the corresponding compilation error information to the business layer, and reads Start the next team. If the compilation is successful, the subprocess first starts to read in the input file and start to execute. During the execution of the subprocess, the parent process will monitor the subprocess at all times. After the program exits normally, it will compare the running results with the expected output file to determine whether they are consistent or meet the requirements of the topic. During the execution of the program, if there is an exception, or some non For example, some malicious code, or the execution time or occupied space of the program reaches the space required by the topic, the parent process will directly kill the child process, and then the parent process will catch the exception information and return it to the business layer. If the program can end normally, the result is consistent with the expected answer, and the program ends successfully in the given time and space of the question, this time is the player's favorite AC (accept). Finally, the kernel writes the corresponding data and information to the database. Multilevel feedback queue scheduling is shown in Fig. 2. Program evaluation flow char is shown in Fig. 3.
When there is only one player, and the frequency of code submission is not very fast, a single processor is fully competent. But in the actual situation, this is unlikely to exist. More importantly, there are multiple players and multiple submission programs. If a single way is adopted, it will make the players wait for some time. If the waiting time is too long, it will severely affect the user experience and reduce the programming enthusiasm of the players. At this time, we adopt the multi process method. First, we keep the code submitted by the user in the database, mark the status as waiting, and tell the user how many jobs are waiting ahead. When the job is scheduled, we tell the user that the program is executing, and generate a progress bar according to the execution time of the program and the given time of the topic. This can optimize some user experience. Because the submitted code cannot write data, so there is no write data conflict. In order to better improve the user experience, multi-processor mode can be used to process multiple jobs in parallel, and multi-level feedback queue scheduling algorithm is used to improve the efficiency.

System Security Control
Microsoft's graphical interface (GUI) has become the global PC market leader so far. Windows is the most popular operating system on the personal computers of college teachers and students. The online judge evaluation system needs to be built on the windows platform, and as a multi-user oriented system, there are insufficient code skills or malicious attacks on the evaluation system. Common malicious behaviors include the following: Dead cycle statements, if not supervised, it will consume CPU system resources; Apply for a large number of useless array space, resulting in stack space overflows, leading to system crash; Obtaining file access rights, trying to modify the files of the key modules of the system to destroy the program operation; Controlling the network, so as to control the dominance of the whole system; so the decision sandbox is the most important part to ensure the normal operation of online judge.
Windows system call is executed by calling system DLL (such as kernel32.dll or gdi32.dll). It is completed by ordinary subroutine call. DLL like kernel32.dll will capture the mechanism of OS privilege layer [11]. Through system call, it refers to the entry point of documented Windows API, such as function createprocess() or function getwindowtext(). Windows consists of two file privileged accounts, system account and administrator account, which have different functions. The system account is used by services running under the windows operating system, among which many services and processes need to be logged in internally. It is an internal account, which is not displayed in the user manager, cannot be added to any group, nor can the user rights be assigned to it. On the other hand, the system account does appear on the NTFS volume in the file manager in the permissions section of the security menu. By default, the system account is granted full control over all files on the NTFS disk, so the system account has the same functional privileges as the administrator account. Access control is to control who can access the security features of the resources in the operating system, and the application program calls the access control function to set who can access the specific resources or control the access to the resources provided by the application program. Through the rlimit() function to monitor the running memory, running time and file handle, which can run the program at the specified time and a certain memory space, it will effectively make use of the system resources to a certain extent. The chroot() function can establish a jail object. The program can only run in the specified directory, which restricts the program to run in the specified directory. In addition, it needs to control the program to open the file. The number of pieces is transferred to pipeline redirection by using standard input. If necessary, set the read-only and non-writable permissions of the user to ensure the security of permissions.
Process Explorer is a process tracking, debugging and monitoring tool in Windows environment. It can display information about which handles and DLL processes have been opened or loaded. The display of the Process Explorer program consists of two sub windows. The top window always displays a list of currently active processes, including the name of the owning account. The information at the bottom of the window depends on the mode of Process Explorer. If it is in processing mode, the processing you will see is selected in the top window. If the process explorer is in DLL mode, you will see the process loading DLL and memory mapping file. Process Explorer also has powerful search capabilities, which can quickly show which processes have opened specific handles or loaded DLLs. Process Explorer's unique features make it useful for tracking DLL version issues or dealing with leaks, and provide insight into how windows and applications work.
At the compilation level, different languages have different compilers. Some languages also correspond to multiple compilers on windows. For example, the famous C language VC++ compiler developed by Microsoft and the MinGW compiler of GNU Project have very strong parameter control and make full use of the parameter function. In addition, there are differences in different languages, when the programs with the same process may run out There is a big gap between them, so it is more appropriate to use the parameter information to calculate the operation time of the program. Moreover, in order to ensure compilation efficiency, direct static compilation, and dynamic compilation may introduce security problems caused by unsafe or incorrect DLL.

Conclusion and Future Work
The system realizes the correct evaluation of the code submitted by the user. The innovation lies in the fact that it can deal with the special questions with different answers. At the same time, the program language supported by the system is rich, which meets the interests of all kinds of students. However, due to the lack of knowledge of the designers, there are still some problems in the implementation of the system, such as the modular design at the beginning of the system design, which cannot reduce the coupling degree between the modules of the system to the minimum, some functions are too powerful, which increases the difficulty of later maintenance. At present, the system has been successfully built on the school server, and has been running smoothly for two months. Through several tests, there are no errors and other abnormalities, and it has been praised by teachers and students alike. In the later stage, the system will continue to be improved and optimized.

Conflicts of Interest:
The authors declare that they have no conflicts of interest to report regarding the present study.