.NET Core下开源任务调度框架Hangfire的Api任务拓展(支持秒级任务)

HangFire的拓展和使用

看了很多博客，小白第一次写博客。

最近由于之前的任务调度框架总出现问题，因此想寻找一个替代品，之前使用的是Quartz.Net，这个框架方便之处就是支持cron表达式适合复杂日期场景使用，以及秒级任务。但是配置比较复杂，而且管理不方便，自己开发了个web管理页面，不过这个需要额外的单独线程去统一管理工作状态，很容易出现问题。

有考虑过 “FluentScheduler” ，使用简单，但是管理配置也很麻烦，我希望能做到配置简单，管理方便，高性能。最后想到了以前听过的hangfire，它的好处就是自带控制面板，在园子里看了很多相关资料，偶然发现了有人拓展过hangfire通过调用api接口来执行任务，这种方式可以避免依赖本地代码，方便部署，在此基础上，我用空闲时间拓展了一下现在已经基本可以满足需求。

所拓展的功能全部属于外部拓展,因此hangfire版本可以一直更新，现在已经更新最新版，支持秒级任务

gitHub地址

由于更新到最新版hangfire 1.7支持秒级任务，使用的在线表达式生成部分表达式有问题,注掉了秒级任务表达式生成，有时间需要详细测试更改,可以参考(hangfire官方提供的表达式)

现在已经实现的功能有：

1，部署及调试：只需要配置数据库连接，然后编译即可运行，无需建表，支持（redis，mysql, sqlserver）其他数据库暂时用不到没测试。推荐使用redis集群。项目中直接添加了redis的存储包，已经更新StackExchange.Redis到最新版本方便拓展，调试时可以直接调试。部署，只需要发布项目，运行创建windows服务的bat命令，命令已经包含在项目中,或者发布至Linux。

2，周期任务：支持在控制面板页面上添加周期任务，编辑周期任务，删除周期任务，手动触发周期任务，暂停和继续周期任务(暂停实现的原理是通过set中添加属性，在job执行前，过滤掉，直接跳过执行，因为hangfire中job一旦创建就失去了控制权，只能通过过滤器去拦截)，任务暂停后会查询状态并渲染面板列表为红色字体方便查找哪个任务被暂停。

3，计划任务：在作业选项卡中，计划作业中可以实现添加计划任务，计划任务可以使任务在指定的分钟后执行，只执行一次。

4，只读面板：通过配置的用户名密码，使用户只具有读取面板的权限，这样可以防止误操作

  //只读面板，只能读取不能操作
             app.UseHangfireDashboard("/job-read", new DashboardOptions
             {
                 AppPath = "#",//返回时跳转的地址
                 DisplayStorageConnectionString = false,//是否显示数据库连接信息
                 IsReadOnlyFunc = Context =>
                 {
                     return true;
                 },
                 Authorization = new[] { new BasicAuthAuthorizationFilter(new BasicAuthAuthorizationFilterOptions
                 {
                     RequireSsl = false,//是否启用ssl验证，即https
                     SslRedirect = false,
                     LoginCaseSensitive = true,
                     Users = new []
                     {
                         new BasicAuthAuthorizationUser
                         {
                             Login = "read",
                             PasswordClear = "only"
                         },
                         new BasicAuthAuthorizationUser
                         {
                             Login = "test",
                             PasswordClear = ""
                         },
                         new BasicAuthAuthorizationUser
                         {
                             Login = "guest",
                             PasswordClear = "123@123"
                         }
                     }
                 })
                 }
             });

5，邮件推送：目前使用的方式是，任务错误重试达到指定次数后，发送邮件通知，使用的MailKit

   catch (Exception ex)
             {
                 //获取重试次数
                 var count = context.GetJobParameter<string>("RetryCount");
                 context.SetTextColor(ConsoleTextColor.Red);
                 //signalR推送
                 //SendRequest(ConfigSettings.Instance.URL+"/api/Publish/EveryOne", "测试");
                 if (count == "")//重试达到三次的时候发邮件通知
                 {
                     SendEmail(item.JobName, item.Url, ex.ToString());
                 }
                 logger.Error(ex, "HttpJob.Excute");
                 context.WriteLine($"执行出错：{ex.Message}");
                 throw;//不抛异常不会执行重试操作
             }

 /// <summary>
         /// 邮件模板
         /// </summary>
         /// <param name="jobname"></param>
         /// <param name="url"></param>
         /// <param name="exception"></param>
         /// <returns></returns>
         private static string SethtmlBody(string jobname, string url, string exception)
         {
             var htmlbody = $@"<h3 align='center'>{HangfireHttpJobOptions.SMTPSubject}</h3>
                             <h3>执行时间：</h3>
                             <p>
                                 {DateTime.Now}
                             </p>
                             <h3>
                                 任务名称：<span> {jobname} </span><br/>
                             </h3>
                             <h3>
                                 请求路径：{url}
                             </h3>
                             <h3><span></span>
                                 执行结果：<br/>
                             </h3>
                             <p>
                                 {exception}
                             </p> ";
             return htmlbody;
         }

邮件模板

  //使用redis
                         config.UseRedisStorage(Redis, new Hangfire.Redis.RedisStorageOptions()
                         {
                             FetchTimeout=TimeSpan.FromMinutes(),
                             Prefix = "{hangfire}:",
                             //活动服务器超时时间
                             InvisibilityTimeout = TimeSpan.FromHours(),
                             //任务过期检查频率
                             ExpiryCheckInterval = TimeSpan.FromHours(),
                             DeletedListSize = ,
                             SucceededListSize =
                         })
                         .UseHangfireHttpJob(new HangfireHttpJobOptions()
                         {
                             SendToMailList = HangfireSettings.Instance.SendMailList,
                             SendMailAddress = HangfireSettings.Instance.SendMailAddress,
                             SMTPServerAddress = HangfireSettings.Instance.SMTPServerAddress,
                             SMTPPort = HangfireSettings.Instance.SMTPPort,
                             SMTPPwd = HangfireSettings.Instance.SMTPPwd,
                             SMTPSubject = HangfireSettings.Instance.SMTPSubject
                         })

配置邮件参数

6，signalR 推送：宿主程序使用的weapi，因此可以通过webapi推送，这样做的好处是可以将服务当作推送服务使用，第三方接口也可以利用此来推送，

  /// <summary>
        ///用户加入组处理
        /// </summary>
        /// <param name="userid">用户唯一标识</param>
        /// <param name="GroupName">组名称</param>
        /// <returns></returns>
         public Task InitUsers(string userid,string GroupName)
         {
             Console.WriteLine($"{userid}加入用户组");
             Groups.AddToGroupAsync(Context.ConnectionId, GroupName);
             SignalrGroups.UserGroups.Add(new SignalrGroups()
             {
                 ConnectionId = Context.ConnectionId,
                 GroupName = GroupName,
                 UserId = userid
             });
             return Clients.All.SendAsync("UserJoin", "用户组数据更新,新增id为：" + Context.ConnectionId + " pid:" + userid);
         }
         /// <summary>
         /// 断线的时候处理
         /// </summary>
         /// <param name="exception"></param>
         /// <returns></returns>
         public override Task OnDisconnectedAsync(Exception exception)
         {
             //掉线移除用户，不给其推送
             var user = SignalrGroups.UserGroups.FirstOrDefault(c => c.ConnectionId == Context.ConnectionId);

             if (user != null)
             {
                 Console.WriteLine($"用户:{user.UserId}已离线");
                 SignalrGroups.UserGroups.Remove(user);
                 Groups.RemoveFromGroupAsync(Context.ConnectionId, user.GroupName);
             }
             return base.OnDisconnectedAsync(exception);
         }

Hub定义

  /// <summary>
         /// 单个connectionid推送
         /// </summary>
         /// <param name="groups"></param>
         /// <returns></returns>
         [HttpPost, Route("AnyOne")]
         public IActionResult AnyOne([FromBody]IEnumerable<SignalrGroups> groups)
         {
             if (groups != null && groups.Any())
             {
                 var ids = groups.Select(c => c.UserId);
                 var list = SignalrGroups.UserGroups.Where(c => ids.Contains(c.UserId));
                 foreach (var item in list)
                     hubContext.Clients.Client(item.ConnectionId).SendAsync("AnyOne", $"{item.ConnectionId}: {item.Content}");
             }
             return Ok();
         }

         /// <summary>
         /// 全部推送
         /// </summary>
         /// <param name="message"></param>
         /// <returns></returns>
         [HttpPost, Route("EveryOne")]
         public IActionResult EveryOne([FromBody] MSG body)
         {
             var data = HttpContext.Response.Body;
             hubContext.Clients.All.SendAsync("EveryOne", $"{body.message}");
             return Ok();
         }

         /// <summary>
         /// 单个组推送
         /// </summary>
         /// <param name="group"></param>
         /// <returns></returns>
         [HttpPost, Route("AnyGroups")]
         public IActionResult AnyGroups([FromBody]SignalrGroups group)
         {
             if (group != null)
             {
                 hubContext.Clients.Group(group.GroupName).SendAsync("AnyGroups", $"{group.Content}");
             }
             return Ok();
         }

推送接口定义

7，接口健康检查：因为主要用来调用api接口，因此集成接口健康检查还是很有必要的，目前使用的方式是配置文件中添加需要检查的地址

 /*健康检查配置项*/
   "HealthChecks-UI": {
     /*检查地址，可以配置当前程序和外部程序*/
     "HealthChecks": [
       {
         "Name": "Hangfire Api 健康检查",
         "Uri": "http://localhost:9006/healthz"
       }
     ],
     /*需要检查的Api地址*/
     "CheckUrls": [
       {
         "Uri": "http://localhost:17600/CityService.svc/HealthyCheck",
         "httpMethod": "Get"
       },
       {
         "Uri": "http://localhost:9098/CheckHelath",
         "httpMethod": "Post"
       },
       {
         "Uri": "http://localhost:9067/GrtHelathCheck",
         "httpMethod": "Get"
       },
       {
         "Uri": "http://localhost:9043/GrtHelathCheck",
         "httpMethod": "Get"
       }
     ],
     "Webhooks": [], //钩子配置
     "EvaluationTimeOnSeconds": , //检测频率
     "MinimumSecondsBetweenFailureNotifications": , //推送间隔时间
     "HealthCheckDatabaseConnectionString": "Data Source=\\healthchecksdb" //-> sqlite库存储检查配置及日志信息
   }

健康检查相关配置

后台会根据配置的指定间隔去检查服务接口是否可以正常访问，(这个中间件可以实现很多检查功能，包括网络，数据库，mq等，支持webhook推送等丰富功能，系统用不到因此没有添加)

健康检查的配置

  //添加健康检查地址
             HangfireSettings.Instance.HostServers.ForEach(s =>
             {
                 services.AddHealthChecks().AddUrlGroup(new Uri(s.Uri), s.httpMethod.ToLower() == "post" ? HttpMethod.Post : HttpMethod.Get, $"{s.Uri}");
             });

健康检查地址添加

  app.UseHealthChecks("/healthz", new HealthCheckOptions()
             {
                 Predicate = _ => true,
                 ResponseWriter = UIResponseWriter.WriteHealthCheckUIResponse
             });
             app.UseHealthChecks("/health", options);//获取自定义格式的json数据
             app.UseHealthChecksUI(setup =>
             {
                 setup.UIPath = "/hc"; // 健康检查的UI面板地址
                 setup.ApiPath = "/hc-api"; // 用于api获取json的检查数据
             });

健康检查中间件配置

其中，ui配置路径是在面板中展示检查结果需要使用的地址

api地址，可以通过接口的方式来调用检查结果，方便在第三方系统中使用，其数据格式可以自定义

通过接口调用

 [{
     "id": ,
     "status": "Unhealthy",
     "onStateFrom": "2019-04-07T18:00:09.6996751+08:00",
     "lastExecuted": "2019-04-07T18:05:03.4761739+08:00",
     "uri": "http://localhost:53583/healthz",
     "name": "Hangfire Api 健康检查",
     "discoveryService": null,
     "entries": [{
         "id": ,
         "name": "http://localhost:17600/CityService.svc/HealthyCheck",
         "status": "Unhealthy",
         "description": "An error occurred while sending the request.",
         "duration": "00:00:04.3907375"
     }, {
         "id": ,
         "name": "http://localhost:9098/CheckHelath",
         "status": "Unhealthy",
         "description": "An error occurred while sending the request.",
         "duration": "00:00:04.4140310"
     }, {
         "id": ,
         "name": "http://localhost:9067/GrtHelathCheck",
         "status": "Unhealthy",
         "description": "An error occurred while sending the request.",
         "duration": "00:00:04.3847367"
     }, {
         "id": ,
         "name": "http://localhost:9043/GrtHelathCheck",
         "status": "Unhealthy",
         "description": "An error occurred while sending the request.",
         "duration": "00:00:04.4499007"
     }],
     "history": []
 }]

接口返回数据原始格式

 {
     "status": "Unhealthy",
     "errors": [{
         "key": "http://localhost:17600/CityService.svc/HealthyCheck",
         "value": "Unhealthy"
     }, {
         "key": "http://localhost:9098/CheckHelath",
         "value": "Unhealthy"
     }, {
         "key": "http://localhost:9067/GrtHelathCheck",
         "value": "Unhealthy"
     }, {
         "key": "http://localhost:9043/GrtHelathCheck",
         "value": "Unhealthy"
     }]
 }

接口返回数据处理后格式

  //重写json报告数据，可用于远程调用获取健康检查结果
             var options = new HealthCheckOptions
             {
                 ResponseWriter = async (c, r) =>
                 {
                     c.Response.ContentType = "application/json";

                     var result = JsonConvert.SerializeObject(new
                     {
                         status = r.Status.ToString(),
                         errors = r.Entries.Select(e => new { key = e.Key, value = e.Value.Status.ToString() })
                     });
                     await c.Response.WriteAsync(result);
                 }
             };

处理方式

8，通过接口添加任务：添加编辑周期任务，添加计划任务，触发周期任务，删除周期任务，多个任务连续一次执行的任务

  /// <summary>
         /// 添加一个队列任务立即被执行
         /// </summary>
         /// <param name="httpJob"></param>
         /// <returns></returns>
         [HttpPost, Route("AddBackGroundJob")]
         public JsonResult AddBackGroundJob([FromBody] Hangfire.HttpJob.Server.HttpJobItem httpJob)
         {
             var addreslut = string.Empty;
             try
             {
                 addreslut = BackgroundJob.Enqueue(() => Hangfire.HttpJob.Server.HttpJob.Excute(httpJob, httpJob.JobName, null));
             }
             catch (Exception ec)
             {
                 return Json(new Message() { Code = false, ErrorMessage = ec.ToString() });
             }
             return Json(new Message() { Code = true, ErrorMessage = "" });
         }

         /// <summary>
         /// 添加一个周期任务
         /// </summary>
         /// <param name="httpJob"></param>
         /// <returns></returns>
         [HttpPost, Route("AddOrUpdateRecurringJob")]
         public JsonResult AddOrUpdateRecurringJob([FromBody] Hangfire.HttpJob.Server.HttpJobItem httpJob)
         {
             try
             {
                 RecurringJob.AddOrUpdate(httpJob.JobName, () => Hangfire.HttpJob.Server.HttpJob.Excute(httpJob, httpJob.JobName, null), httpJob.Corn, TimeZoneInfo.Local);
             }
             catch (Exception ec)
             {
                 return Json(new Message() { Code = false, ErrorMessage = ec.ToString() });
             }
             return Json(new Message() { Code = true, ErrorMessage = "" });
         }

         /// <summary>
         /// 删除一个周期任务
         /// </summary>
         /// <param name="jobname"></param>
         /// <returns></returns>
         [HttpGet,Route("DeleteJob")]
         public JsonResult DeleteJob(string jobname)
         {
             try
             {
                 RecurringJob.RemoveIfExists(jobname);
             }
             catch (Exception ec)
             {
                 return Json(new Message() { Code = false, ErrorMessage = ec.ToString() });
             }
             return Json(new Message() { Code = true, ErrorMessage = "" });
         }
         /// <summary>
         /// 手动触发一个任务
         /// </summary>
         /// <param name="jobname"></param>
         /// <returns></returns>
         [HttpGet, Route("TriggerRecurringJob")]
         public JsonResult TriggerRecurringJob(string jobname)
         {
             try
             {
                 RecurringJob.Trigger(jobname);
             }
             catch (Exception ec)
             {
                 return Json(new Message() { Code = false, ErrorMessage = ec.ToString() });
             }
             return Json(new Message() { Code = true, ErrorMessage = "" });
         }
         /// <summary>
         /// 添加一个延迟任务
         /// </summary>
         /// <param name="httpJob">httpJob.DelayFromMinutes（延迟多少分钟执行）</param>
         /// <returns></returns>
         [HttpPost, Route("AddScheduleJob")]
         public JsonResult AddScheduleJob([FromBody] Hangfire.HttpJob.Server.HttpJobItem httpJob)
         {
             var reslut = string.Empty;
             try
             {
                 reslut = BackgroundJob.Schedule(() => Hangfire.HttpJob.Server.HttpJob.Excute(httpJob, httpJob.JobName, null), TimeSpan.FromMinutes(httpJob.DelayFromMinutes));
             }
             catch (Exception ec)
             {
                 return Json(new Message() { Code = false, ErrorMessage = ec.ToString() });
             }
             return Json(new Message() { Code = true, ErrorMessage = "" });
         }
         /// <summary>
         /// 添加连续任务,多个任务依次执行，只执行一次
         /// </summary>
         /// <param name="httpJob"></param>
         /// <returns></returns>
         [HttpPost, Route("AddContinueJob")]
         public JsonResult AddContinueJob([FromBody] List<Hangfire.HttpJob.Server.HttpJobItem> httpJobItems)
         {
             var reslut = string.Empty;
             var jobid = string.Empty;
             try
             {
                 httpJobItems.ForEach(k =>
                 {
                     if (!string.IsNullOrEmpty(jobid))
                     {
                         jobid = BackgroundJob.ContinueJobWith(jobid, () => RunContinueJob(k));
                     }
                     else
                     {
                         jobid = BackgroundJob.Enqueue(() => Hangfire.HttpJob.Server.HttpJob.Excute(k, k.JobName, null));
                     }
                 });
                 reslut = "true";
             }
             catch (Exception ec)
             {
                 return Json(new Message() { Code = false, ErrorMessage = ec.ToString() });
             }
             return Json(new Message() { Code = true, ErrorMessage = "" });
         }

通过接口添加任务

这样做的好处是有效利用了宿主的webapi，而且无需登录控制面板操作就能实现任务管理，方便集成管理到其他系统中

防止多个实例的任务并行执行，即一个任务未执行完成，另一个相同的任务开始执行，可以使用分布式锁来解决

通过特性来添加任务重试时间间隔(hangfire 1.7 新增，单位/秒)，重试次数，队列名称，任务名称，以及分布式锁超时时间

 /// <summary>
         /// 执行任务，DelaysInSeconds(重试时间间隔/单位秒)
         /// </summary>
         /// <param name="item"></param>
         /// <param name="jobName"></param>
         /// <param name="context"></param>
         [AutomaticRetry(Attempts = , DelaysInSeconds = new[] { , ,  }, LogEvents = true, OnAttemptsExceeded = AttemptsExceededAction.Fail)]
         [DisplayName("Api任务:{1}")]
         [Queue("apis")]
         [JobFilter(timeoutInSeconds: )]

配置分布式锁超时时间

 //设置分布式锁,分布式锁会阻止两个相同的任务并发执行，用任务名称和方法名称作为锁
             var jobresource = $"{filterContext.BackgroundJob.Job.Args[1]}.{filterContext.BackgroundJob.Job.Method.Name}";
             var locktimeout = TimeSpan.FromSeconds(_timeoutInSeconds);
             try
             {
                 //判断任务是否被暂停
                 using (var connection = JobStorage.Current.GetConnection())
                 {
                     var conts = connection.GetAllItemsFromSet($"JobPauseOf:{filterContext.BackgroundJob.Job.Args[1]}");
                     if (conts.Contains("true"))
                     {
                         filterContext.Canceled = true;//任务被暂停不执行直接跳过
                         return;
                     }
                 }
                 //申请分布式锁
                 var distributedLock = filterContext.Connection.AcquireDistributedLock(jobresource, locktimeout);
                 filterContext.Items["DistributedLock"] = distributedLock;
             }
             catch (Exception ec)
             {
                 //获取锁超时，取消任务，任务会默认置为成功
                 filterContext.Canceled = true;
                 logger.Info($"任务{filterContext.BackgroundJob.Job.Args[1]}超时,任务id{filterContext.BackgroundJob.Id}");
             }

过滤器添加分布式锁

  if (!filterContext.Items.ContainsKey("DistributedLock"))
             {
                 throw new InvalidOperationException("找不到分布式锁，没有为该任务申请分布式锁.");
             }
             //释放分布式锁
             var distributedLock = (IDisposable)filterContext.Items["DistributedLock"];
             distributedLock.Dispose();

释放分布式锁

通过过滤器来设置任务过期时间，过期后自动在数据库删除历史记录

 public void OnStateApplied(ApplyStateContext context, IWriteOnlyTransaction transaction)
         {
             //设置过期时间，任务将在三天后过期，过期的任务会自动被扫描并删除
             context.JobExpirationTimeout = TimeSpan.FromDays();
         }

设置任务过期时间

redis集群下，测试秒级任务

集群为windws环境下，一个主节点四个从节点，(使用时需要在redis连接中配置全部集群连接，主节点和从节点)，目前用不到linux环境，没有进行测试。