Upload New File

人工智能系统实战第三期/实战代码/计算机视觉/FCN&UNet/dataPrepare.ipynb

+{
+{
+ "cells": [
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from data import *"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# data augmentation \n",
+    "\n",
+    "In deep learning tasks, a lot of data is need to train DNN model, when the dataset is not big enough, data augmentation should be applied.\n",
+    "\n",
+    "keras.preprocessing.image.ImageDataGenerator is a data generator, which can feed the DNN with data like : (data,label), it can also do data augmentation at the same time.\n",
+    "\n",
+    "It is very convenient for us to use keras.preprocessing.image.ImageDataGenerator to do data augmentation by implement image rotation, shift, rescale and so on... see [keras documentation](https://keras.io/preprocessing/image/) for detail.\n",
+    "\n",
+    "For image segmentation tasks, the image and mask must be transformed **together!!**"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## define your data generator\n",
+    "\n",
+    "If you want to visualize your data augmentation result, set save_to_dir = your path"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "#if you don't want to do data augmentation, set data_gen_args as an empty dict.\n",
+    "#data_gen_args = dict()\n",
+    "\n",
+    "data_gen_args = dict(rotation_range=0.2,\n",
+    "                    width_shift_range=0.05,\n",
+    "                    height_shift_range=0.05,\n",
+    "                    shear_range=0.05,\n",
+    "                    zoom_range=0.05,\n",
+    "                    horizontal_flip=True,\n",
+    "                    fill_mode='nearest')\n",
+    "myGenerator = trainGenerator(20,'data/membrane/train','image','label',data_gen_args,save_to_dir = \"data/membrane/train/aug\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## visualize your data augmentation result"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Found 30 images belonging to 1 classes.\n",
+      "Found 30 images belonging to 1 classes.\n"
+     ]
+    }
+   ],
+   "source": [
+    "#you will see 60 transformed images and their masks in data/membrane/train/aug\n",
+    "num_batch = 3\n",
+    "for i,batch in enumerate(myGenerator):\n",
+    "    if(i >= num_batch):\n",
+    "        break"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## create .npy data\n",
+    "\n",
+    "If your computer has enough memory, you can create npy files containing all your images and masks, and feed your DNN with them."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "image_arr,mask_arr = geneTrainNpy(\"data/membrane/train/\",\"data/membrane/train/\")\n",
+    "np.save(\"data/image_arr.npy\",image_arr)\n",
+    "np.save(\"data/mask_arr.npy\",mask_arr)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "outputs": [],
+   "source": [],
+   "metadata": {
+    "collapsed": false
+   }
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.6.2"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}