Umar Khan

Machine Learning Engineer and Researcher at DFKI, working on Computer Vision and NLP based systems with Dr. Sheraz Ahmed. My previous works include working with Dr.Faisal Shafait on various Computer Vision R&D and commercial Projects, since my undergrad I have also had a keen focus on Requirements Engineering with my work for aawaz , which proceeded to winning the IBM Pitch your startup cup.

with keen focus on application of research I have worked on numerous commercial projects, you can read more about them here

Interests: Computer Vision, Graph Neural Networks, Autonomous Systems, Biomimicry.

Publications

• TabAug: Data Driven Augmentation for Enhanced Table Structure Recognition Umar Khan, Sohaib Zahid, Muhammad Asad Ali, Adnan ul Hassan, Faisal Shafait [paper] Publication Status: Accepted at ICDAR2021
• Content-Driven Document Colorization for Table Detection Muhammad Asad Ali, Sohaib Zahid, Umar Khan, Adnan ul Hassan, Faisal Shafait [pre-print] Publication Status: Pre-print

Notable Projects

• DeepReader AI Data Extraction Automation Services Currently working on following projects:
  1. Using GNN's for NER.
  2. Table Detection and Recognition using Lightweight Models.
  3. Implementing Custom OCR model and lifecycle for constant improvement.
  4. Deep Learning based perspective transformations for data extraction services.

• DeepReader AI Data Extraction Automation from German Language Documents Trained state of the art NER for German Documents, we've seen a massive improvements in results by employing different strategies for training and given the green light for open sourcing from the company I might document my experimentation in my blog.
• PhelixAI Data Extraction Automation in Medical Insurance Documents Developed a Flask Based WEB API for Medical Document Classification, Medical Object Detection and Medical NER for Automatic Data Filling, We also developed an in house standard for Data Storage, Training Procedures from Database stored Ground Truths and Inference Pipeline. The Medical Document NER was a custom model architecture based on LayoutLM base model architecture and outperformed all the existing models.
• TabAUG WebAPP(Active) Building a Full Scale WebAPP for TabAUG for demonstration The Project contains developing a full scale webapp for processing tables to generate augmented tables. The webapp would also provide the users with their existing tables metadata and augmented tables metadata. Update: Statistics Generation features are all designed and implemented, Augmentation To follow.
• Pneumonia Detection [website] tested and implemented a new form of pre-processing white exposure limits to detect pneumonia, achieved a 92.76% testing accuracy, created a webapp for a client test, currently live on the link given above. Note: The advises generated are static based on various symptom inputs and confidence score of the algorithm.

• FishTally Fish Detection and Counting in Underwater Environment Extending upon Ahsan Jalal's work on fish recognition, my work includes working on combining GMM (gaussian mixture model) background substracted image features with optical flow image features to ehance the original image to be fed into yolov3 darknet and Faster RCNN for fish recognition in video sequences. Another planned addition is utilizing Tracktor++ to track and count species of fishes in a video sequence.
• SynthGen Synthetic Tabular Data Generation For Controlling Model Behavior using UNLV dataset format I am building a tool which allows the user to read their data pool to analyze different charactaristics of their data and then generate more data of similar charactaristics or change them as per their requirement, we're currently testing controlling model behavior with generated data by conducting experiments on Deep Splitting and Merging model.
• Tabsegv2 Table Structure Decomposition with SPLIT model I worked on developing an end to end pipeline for extracting data from scanned table images, We used the earlier developed TableDet detection module using Faster RCNN to detect tables and then used Split Model along with merge heuristics to detect row/column segmenations, once segmented cells were produced and then exported as XML and CSV formatted data.
• RecAARZ Recommendation Engine for AARZ built an API atop Recommendation Engine for a real estate buy and sell platform known by the name of AARZ. We used the matrix factorisation model known as LightFM for building the Recommendation Engine as it outperformed alternative methods in both cold start and sparse interaction matrix scenario.

• TableDet Table Detection in Scanned Document Images [blog] Tables having very little similarity to natural images, hence they were first preprocessed with distance transform in a way to build resemblance to natural images so that we could utilize pretrained ResNet50 backbone, these preprocessed images were then served to a luminoth based Faster RCNN. Starting accruacy achievable with publicaly available models was 52 percent, after feature engineering this figure was increased to 78 percent with >90% IOU and 22 percent partial detection with >80% IOU, with only 3% missed tables with <20% IOU.
• TabSegv1 Table Structure Decomposition with GNN's worked on GNN's (Graph Neural Networks) for decomposing trabular data structure for data extraction autmation in Tabular Data. Proposing a new idea of OCR data closing (OCR bounding box dilation and erosion) for constructing spatial sentences for improving accuracy on CloudSDS Proprietery Dataset.
• Object Concurrency Checking Object Concurrency in Point Cloud Data [Details] Developed a tool for object concurrency in construction process, with real data as a point cloud data and planned data from BIM4D file as an object, we used python OCC library for checking point cloud density inside BIM4d Objects to classify BIM4d objects as existing or non existing in point cloud data. the prototype had a 100% accuracy.
• QuickAnnot Horizontal Vertical Annotations in Tables [Details] Part of Data extraction in tables depend upon where the headings of the tables are situated, defining Horizontal and Vertical tables becomes increasingly important, I developed an annotation tool in Tkinter for UNLV format with a direct focus on speed.
• TransferLight Transfering Highlights from Source PDF to Target PDF A tool for detecting highlighted codes in source PDF document and transferring highlights to a targer PDF document, I used open source libraries PDFMiner and PyPDF2 to recursively traverse PDF Objects to find highlights inside the Source pdf documents and then transfer them in similar methodology to Target pdf documents. The tool was developed for Vactor Manafacturing Inc, prior to the tool the company spent 15 minutes per document processing with manual labor, the tool spent an average 21 seconds per document processing getting it closer to maximizing processing potential.