Automated Power Pole Photography via Helicopters

Automated Power Pole Photography via Helicopters Fugro Roames aims to automate the process of acquiring high resolution pictures, to reduce the risk and cost associated with helicopter based pole top inspection. The project will focus on acquiring previously worked algorithms, as well as the introduction of model predictive control, to automate the overall process, while also presenting a working prototype to simulate the operation of the intended product. The following proposal will aim to describe the intended topic and scope, while also reviewing background information on aerial asset management and autonomous aerial photography related to the project. The report will also highlight all achievable milestones, and their respective tasks, within the project plan; as well as present a detailed OHS risk assessment of the projects practical and non-practical work. The work completed during the semester, will hope to provide sufficient groundwork for automated aerial asset management procedures, within the electrical distribution industry. Accurate and effective asset condition management is important to ensure the longevity of an electric distribution network, while maximising its performance and operational efficiency.[1] At present, an efficient and cost effective method to test the integrity of a network, requires the use of a human-piloted helicopter and a photographer to capture high resolution images of power poles and their subcomponents [2]. Fugro-Roames, a company which currently provides this to its customers, aims to reduce the risk and cost associated with helicopter based pole top inspection, by replacing the photographer with an automated camera gimbal to capture high resolution pictures of the network. Figure 1.1: Power Pole Photography [1] Automated aerial asset management in the context of this proposal refers to the use of aerial platforms, to asses specific assets in an easy and cost-efficient manner, without the need of manual involvement. Unfortunately, and as it will be discussed in Chapter 3, the project topic is a form of technology that has not been widely researched; however, various methods of aerial asset management, such as helicopters, UAVs or drones, do exist and are currently in use within the industry. 2.1 Project Outline It is the purpose of this project to develop a Receding Horizon Planner (RHP) in order to automate power pole photography using camera gimbal system. The RHP will based on the algorithms created by Dr. Michael Kearney, which provide a solution to the photograph feasibility, photograph scheduling, and gimbal trajectory planning problems [3]. For validation, the developed planner will be tested using simulation flight data provided by Fugro-Roames. Further testing will include the implementation of the RHP into a gimbal prototype, where sensitivity analyses and assessment of the initial assumptions will be completed to determine the projects limitations and outline recommendations for future work. A more defined project plan will be described in Chapter 4 below. 2.2 Motivation for Automated Aerial Asset Management In order to overcome the limitations associated with conventional asset management methods, a high resolution image capture system was developed and is now used to aid in the inspection, assessment and maintenance of electric distribution networks [4]. However, the overall cost associated with this method, outweights its improved efficiency, as it involves specialized labour (photographers) and the use of helicopters to complete the required task [5]. The motivation to introduce an automated aerial asset management system, requires for the reduction of risks and overall cost associated with the current model, as well as improving the quality and selection of photographs taken. 2.3 Project Aim, Objectives Intended Scope Since the project, and all information available, are sponsored and provided by FugroRoames, the aims and objectives have been defined by the companys desires for the finished product. Therefore, the aim and proposed purpose of the project is to reduce the cost and risk associated with helicopter based pole top inspection, with the use of a Receding Horizon Planner (RHP) that automates the movement and control of a camera gimbal system. Along with the proposed project aim, multiple objectives must also be met whilts completing the work required. These include improving the algorithms created by Dr. Kearney, obtaining accurate efficiency gains for the RHP, designing and building a working gimbal prototype to be used for testing, and providing sufficient ground work for actual on-site testing and implementation, with the use of a helicopter, of the RHP beyond the project. Similar to the projects aim and objectives, the intended scope has also been shaped by the companys desires for the final product. Therefore, the scope can be outlined as: Review of background information and related work Adaptation of algorithms created by Dr. Kearney Development of Receding Horizon Planner Design and build of a gimbal prototype Planner implementation and testing Analysis of results Sensitivity analysis Assesment of projects assumptions Evaluation of project and suggestion for future work Possible gimbal rig implementation and on-site testing Taking into account the scope described above, it seems logical to break the project into three specific sections; planner development and testing, prototype design and testing, and thorough result analysis. The development and testing of the Receding Horizon Planner involves the improvement of existing control algorithms, to implement and verify its overall efficiency, using available and provided data. The design and testing of the gimbal prototype, which should resemble the actual gimbal rig, involves the use of the developed Receding Horizon Planner to validate and improve previously obtained results. Finally, thorough result analysis requires the breakdown of the planner and obtained results, to find how assumptions, parameters and particular components were affetcted. Although automated aerial asset management is a form of technology that has not been widely researched, the following chapter will provide a complete review of background literature which would closely resemble the general subject matter. The review will be broken into two sections; aerial asset management, and autonomous aerial photography and gimbal control. Previous work related to these topics will be presented, reiterated and reviewed, focusing on sources related to asset management within the electric power distribution industry. 3.1 Aerial Asset Management Aerial asset management , within the electric power distribution industry, has been implemented to replace conventional asset management and inspection methods, and provide a fast and accurate way to determine any defects that could be present. Whitworth et al. [6], in a work sponsored by EA Tecnology, propose the use of a helicopter-mounted camera to capture and store visual information, in order to enhance the inspection of overhead power lines. In order to reduce camera shake and partially automate the inspection process, the authors recommend the use of an acquisition system, which finds and locks the camera to the location of the powerline, followed by a recursive algorithm that tracks the powerline smoothly, despite the translation of the helicopter. Similarly, Earp et al. [1][4] describe an aerial inspection technique, which was also developed by EA Technology, that uses high resoluion images to perform a detailed condition assessment on electrical towers within a distribution network. The authors break down the helicopter based condition assessment, which is considered an improvement from the video inspection method in [6], to include four different parts: Pre-flight Planning: Inspection requirements, photographic sequences, camera trajectories; as well as current wind and weather conditions, natural and man-made ground feature, and the locations of the electrical towers, are all taken into account during the pre-flight planning. Helicopter Inspection and Picture Acquisition: A high resolution digital camera is used to take a set number of images, per tower, to meet the inspection requirements. Satellite-based Global Positioning System (GPS) and moving map displays are used to georeference each photograph taken, back to the tower. Image Processing, Analysis and Condition Assessment: Captured images are examined and given a Condition Rating (CR), typically on a scale of 1 to 4 (1 describing best condition, 4 describing worst condition). The uniformity of the assessment, determined by the individual DNOs requirements, is ensured by this critical step and therefore requires for a detailed condition assessment criteria, application-specific workstations, and accurate in-house training programme for assessors. Condition Based Risk Management (CBRM): A process developed by EA Technology, it combines practical and theoretical knowledge about a specific asset, along with maintenance experience, in order to define its current condition. Taking a different approach, N. Ellis [7] investigates the use of Unmanned Aerial Vehicles (UAVs) to inspect power transition lines. The author investigated the cost, risks and overall efficiency that comes with the use of UAVs, searching for low budget automation strategies to be designed and tested. Unfortunately, due to government regulations and the high capital and operating cost of the UAV, lead the author to the conclusion that the technology is not feasible at the current time. As outlined by most of the sources presented, the introduction of aerial asset management techniques, has made a big improvement on the inspection, assessment and maintenance of an electrical distribution network. Whilst most models present techniques that far surpass conventional inspection methods, the cost that comes with the involvement of specialized labour and helicopters, leaves little room for errors and inconsistent results. However, although the implementation of an UAV was not possible due to the introduction of new risks, the automation techniques, presented in [7], can be applied to previously discussed aerial photography techniques, and mitigate/remove any currently involved risks. 3.2 Autonomous Aerial Photography and Gimbal Control Autonomous aerial photography and control, within the electric power distribution industry, is not a topic that has been widely researched or implemented. However, the use of a camera and aerial images to predict and control the movement of UAVs is something that is commonly discussed and will therefore, be the main focus for this section. E. Skjong et al. [8] investigate the recent commercial availability of UAVs within Search and Rescue (SAR) and Search and Tracking (SAT) applications. The authors then focus on the development of a SAT system, which is able to steer the UAV and focus the gimbal attitude on regions and objects of interest respectively, with the use of Model Predictive Control (MPC). The overall process is made autonomous by allowing computer vision to work directly with the UAV autopilot and MPC, so objects can be simultaneously detected and tracked in an efficient manner. Similarly, C.E. Lin and S. Yang [9] explore the use of UAVs to detect and track specific objects, with the help of aerial photography and camera gimbal control. The authors implement the use of an Inertial Measurement Unit (IMU), which consist of a gyroscope, accelerometer, and magnetometer, along with an Attitude and Heading Reference System (AHRS), to determine and ensure that the angles of the camera gimbal are in the correct reference frame. Both [8] and [9] use Global Positioning System (GPS) to determine the location of both the UAV and the target, using this relationship to implement a reliable autopilot flight control for target detection and photography. R.J. Rajesh and C.M. Ananda [10] move away from controlling the camera gimbal, attached to a UAV, and focus on stabilizing its movement to ensure that clear photograph and/or video footage is taken. The use of Proportional-Integral-Derivative (PID) controller is recommended by the authors, to compensate for the vibrations and gust, as well as control the position of the camera by stabilizing the movement of the gimbal. Manually tuning the controllers parameters is not recommended, as the process is considered time consuming and tedious, instead, the authors recommend the use of Particle Swarm Optimization (PSO) as the preferred algorithm to complete this task. Uncertainty and disturbances are mentioned, but not properly investigated in [9], [8] and [10]. A. Ashok et al. [11] investigate the external disturbances that affect the UAVs, as well as the dynamic and parametric uncertainties that arise in the mathematical autonomous model when subjected to a number of operating conditions. The authors reiterate previous approaches taken to design a robust control system, including the use of a PID controller for linear [12] and linearized [13] models, as well as the use of a Linear Quadratic Gaussian (LQG) controller [14] in the presence of uncertainties, before the Uncertainty and Disturbance Estimation (UDE) method is chosen to synthesize the required controller. The control of a camera gimbal, as outlined by most of the sources above, is necessary in order to ensure the accuracy of photographs or video that is captured by the UAV. Although the use of conventional control methods is described above, only [8] focuses on the use of MPC, which is closely related to the project, to ensure that the UAV is able to detect and track objects efficiently and simultaneously. A clear representation of the projects tasks, has been outlined as a comprehensive project plan from the first to the final day of employment at Fugro-Roames. A visual representation and description of the plan is used to illustrate the timeline of the project, including all achievable milestones, which are related to the aims, objectives and intended scope of the project, discussed in section 1.2. 4.1 Visual Representation of Project Plan The use of a modified GANTT chart was implemented to showcase the proposed flow of the project, from the first to the last day of employment. The timeline is hence separated into 24 weeks that are broken into 5 days, in the same manner as the business week format, where the project milestones and their corresponding tasks are allocated a precise number of days in which work is scheduled to take place. Green solid bars represent the projects milestones, where red solid bars indicate their respective tasks. The progress of the overall project is tracked by the completion of every achievable milestone, which can only be completed by first completing their respective sub tasks. Completed milestones are shown with a blue line through the green bar, and completed tasks are shown with a yellow line through the red bar. Weeks 1 to 12, as shown in Figure 4.1, represent the core work to be completed, as Milestones 3-5 directly relate to the aims, objectives and intended scope of the project. Weeks 13 to 22, as shown in Figure 4.2, outline the analysis and completion of the final pieces of assessment, including the thesis report and demonstration, which require the content from previous milestones to be completed. Weeks 23 to 24, also shown in Figure 3.2, outline a possible Milestone that can be completed until the final day of employment at Fugro-Roames. This milestone does not affect the previous pieces of assessment; however, it will provide the company with important information that could prove vital to the future of the project. 4.2 Milestone Task Breakdown Milestone 1: Project Scope The project scope is necessary to determine the projects main objectives, and will therefore guide the work to be completed throughout the semester. To ensure all expectations are met, and the appropriate time is given to all project milestones, an agreement between academic and industry supervisors is necessary. Task ID Days Description Project evaluation 1.1 5 Evaluate the requirements of the project, including potential goals and outcomes. Collect all necessary information to present during the supervisor meeting. Supervisor meeting 1.2 1 Meet with the projects academic and industry supervisors to discuss the project scope, and agree on the due dates for all pieces of assessment. Scope confirmation agreement 1.3 2 Compile a detailed scope which highlights the projects objectives, as discussed during the supervisor meeting. Resources Workspace with an available computer Availability from both supervisors to organize a meeting Milestone Hazards and Risks Milestone 1 is essential to the project. Any delay could be considered a minor, but possible risk, as it affects the progress of the overall project. If this issue becomes bigger, and the project scope cannot be defined, then it can be classified as a major risk and mitigation strategies should be taken immediately. Clear communication between all parties involved, can reduce and remove the possibility of the identified risks from occurring. Milestone 2: Project Proposal The project proposal is the first piece of assessment, which must be completed as part of the responsibilities for completing the project placement. The report highlights the work from Milestone 1, as it expands on the agreed aims and objectives, outlines the background information related to the projects main topics, and presents a visual and a clear representation of the project plan. A detailed OHS risk assessment, which analyses the potential risk involved with the projects practical work, and any potential equipment used, will also be included in the proposal. Task ID Days Description Research of relevant material 2.1 5 Find background information and prior art related to the projects main topic. Project outline and intended scope 2.2 4 Expand on the projects scope decided in Task 1, expanding on the projects aims and objectives. Background and related work 2.3 3 Summarize and expand on the material obtained from Task 2.1. Project plan summary 2.4 3 Provide a detailed project plan, with logically ordered tasks and their respective milestones. OHS risk assessment 2.5 2 Compile an OHS risk assessment, which highlights the projects practical work and equipment used. Drafting and submission 2.5 5 Assemble the proposals individual sections, review the written report and submit via Turnitin. Resources Completion of Milestone 1 Confirmation of university assessment due dates OHS risk assessments, regarding the use and control of the gimbal rig Milestone Hazards and Risks The project proposal relies mostly on individual and previously completed work, however, certain sections require resources which are not readily available. The most significant obstacles, which would require mitigation, are the confirmation of all university assessment due dates and the risk assessments completed for the gimbal rig, which might be used during the project. Clear and constant communication with the universitys course coordinator, as well as the personnel in charge of the gimbal rig, is essential to mitigate and prevent any issues that could affect the completion of the project proposal. Milestone 3: Receding Horizon Planner Milestone 3 is the first milestone that uses the algorithms created by Dr. Kearney to develop an on-line planner that controls the photograph scheduling plan and the movement of the gimbal throughout the event horizon. The completed Receding Horizon Planner will involve the use of a low level controller, an upper level planner, and an event monitor, to be used in Milestones 4à ¢Ã¢â‚¬  Ã¢â‚¬â„¢7. Task ID Days Description Algorithm testing and improvement 3.1 5 Improve the current photograph feasibility algorithm, and test its efficiency. Model predictive control research 3.2 5 Find related material to be used when designing the Receding Horizon Controller. Lower level controller design implementation 3.3 5 The gimbal and camera are robustly controlled by the photograph scheduling algorithm chosen by the upper level planner. Upper level planner design implementation 3.4 5 Implemented the improved algorithm from Task 3.1 to generate a plan that the system will follow over a prediction horizon. Event monitor design implementation 3.5 4 Connect the results from Tasks 3.3 and 3.4 so that the plan is implemented and changed after certain events occur. Resources Programming and modelling software Access to the projects repository and previous work Milestone Hazards and Risks The progress of Milestone 3 could be significantly affected, if access to the necessary repositories and previous work is delayed. As previous algorithms are necessary to the development of the Receding Horizon Planner, the completion of the project would be significantly impacted. Clear communication with the right personnel can help mitigate this issue before it affect the timeline and progress of the project. Milestone 4: Planner Analysis and Improvement Rigorous testing of the developed Receding Horizon Planner is required to find the necessary efficiency gains, so the planner can be implemented on the projects physical equipment. Data obtained from Fugro Roames, as well as the analysis and reiteration of the testing completed, will be completed to improve the found controller gains. Task ID Days Description Test current and new scenarios 4.1 4 Implement the Receding Horizon Planner on several simulated scenarios, using data received from Fugro Roames. Reiteration of Receding Horizon Planner testing 4.2 3 Fix any errors found in Task 4.1 and repeat the tests. Determine accurate efficiency gains 4.3 3 Determine the required controller gains which provide the most accurate results. Resources Completion of Milestone 3 Sufficient testing data provided by Fugro Roames Milestone Hazards and Risks The lack of testing data used to complete this milestone, is a minor risk that could affect the project. Requesting said data ahead of time, would ensure that it is ready for when testing of the Receding Horizon Planner begins, leaving the project timeline unaffected. Milestone 5: Gimbal Prototype Milestone 5 marks a key point in the project, as the implementation of the Receding Horizon Planner on a working prototype is essential to the projects success. The design of the prototype will be based on the actual gimbal rig owned and created by FugroRoames, to facilitate the implementation of the planner for further testing, at the end of the project. Task ID Days Description Embedded system design and build 5.1 7 Design, build and combine the mechanical, electrical, and software components of the prototype. Implementation of Receding Horizon Planner 5.2 6 Test the Receding Horizon Planner using the gimbal prototype. Sensitivity Analysis 5.3 5 Identify and vary the dominant parameters, testing and improving the Receding Horizon Planner where possible. Resources Completion of Milestone 4 Mechanical, electrical and software design software Working space and necessary build equipment Development of testing scenarios Milestone Hazards and Risks Milestone 5 introduces the use of practical equipment to design, test and build each component of the gimbal prototype. The misuse of the practical equipment, and the lack of component testing, are immediate risks to the completion of the prototype. Following the risk assessment outlines on Chapter 4, as well as completing the required testing before the Receding Horizon Planner is implemented, can help mitigate the risks described and prevent possible delays. Milestone 6: Assessment of Projects Assumptions As previously shown by Dr. Kearney, on the initial report he completed for Fugr-Roames, the introduction of the Receding Horizon Planner required changes to the initial assumptions made when designing the projects photograph allocation and gimbal control algorithms. The change and/or the addition of assumptions, by analysing the projects missing parameters is, therefore, also essential when validating the results obtained in Milestone 5. Task ID Days Description Identify missing parameters 6.1 2 Determine the projects missing parameters based on the assumptions made by Dr. Kearney on his report. Evaluate the effects of missing parameters 6.2 3 Assess how the model is affected by each missing parameter. Update the projects assumptions 6.3 2 Compose a list of updated assumptions based on the results from tasks 6.1 and 6.2, to be used on further work. Resources Completion of the Sensitivity Analysis Task within Milestone 5 Milestone Hazards and Risks The risks to Milestone 6 are relatively small, where the largest possible problem involves the inability to identify the effects of the projects missing parameters. Keeping a record of all parameter changes/assumptions made during previous tests is the best solution to mitigate/prevent this issue. Milestone 7: Project Evaluation and Recommendation for Future Work Identifying the projects limitations is an important process, as it allows recommendations for future work to be made and included on the final thesis report. Task ID Days Description Identify projects limitations 7.1 3 List the projects limitations, based on the results from Task 6. Develop accurate recommendations 7.2 3 Expand on previous work and results from Tasks 6.1 to 6.3, including 7.1, to develop accurate recommendations for future work. Resources Development of the Receding Horizon Planner Sensitivity analysis of the gimbal prototype Completion of Milestone 6 Milestone Hazards and Risks The completion of Milestone 7 is only affected by the work done in previous milestones. If previous tasks are delayed, any future recommendations run the risk of not being completed. To prevent this from affecting the final stages of the project, the plan/timeline outlined should always be followed, noting wher