Fixinโ€™ ResNet / ๋ ˆ์Šค๋„ท ๊ณ ์น˜๊ธฐ

Diggin’ into neural nets.

A ResNet is a relatively new type of deep artificial neural networks that can recognize objects – the kind that finds objects in your images on Google Photos.

This illustration mixes images of a stack of sandwiches with a ResNet. The protagonist is adding a “skip connection” (blue wire) between neural layers (slices of bread). Such skip connections, inspired by biology, are the characteristic of ResNets. The skip connections allowed ResNets to be deeper than its predecessors (have a tall stack), and helped them to recognize more complex images.

Why is it good for a network to be deep? And why do skip connections help? I find it a stretch to explain them with the analogy of a sandwich, so I defer the answers to later posts. Let’s say for now that the ResNet has skip connections repeated every 2-3 layers, so it’s easy to make it deeper by stacking the same structure multiple times. That makes it look like the stacked sandwiches (see Figure 3 of the original paper).

Well, this was my first attempt at a neuroscience/AI-inspired illustration. The analogy leaves a lot to be desired, but hopefully it will get better as I try more. At least this fulfills my new year’s resolutionโ€”to post about neuroscienceโ€”on the new year’s day! ๐Ÿ™‚

์‹ ๊ฒฝ๋ง ์† ํŒŒ๊ณ ๋“ค๊ธฐ.

๋ ˆ์Šค๋„ท์€ ๋น„๊ต์  ์ƒˆ๋กœ์šด ์‹ฌ์ธต ์‹ ๊ฒฝ๋ง์œผ๋กœ, ๊ตฌ๊ธ€ ํฌํ† ์—์„œ์ฒ˜๋Ÿผ ๋ฌผ๊ฑด๋“ค์„ ์•Œ์•„๋ณผ ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.

๊ทธ๋ฆผ์—์„œ๋Š” ๋†’์ด ์Œ“์€ ์ƒŒ๋“œ์œ„์น˜์™€ ๋ ˆ์Šค๋„ท์˜ ์ด๋ฏธ์ง€๋ฅผ ํ•ฉํ–ˆ์Šต๋‹ˆ๋‹ค. ์ฃผ์ธ๊ณต์€ “๊ฑด๋„ˆ๋›ฐ๋Š” ์—ฐ๊ฒฐ”(ํŒŒ๋ž€ ์ „์„ )์„ ์‹ ๊ฒฝ๋ง ์ธต (๋นต) ์‚ฌ์ด์— ์ถ”๊ฐ€ํ•˜๊ณ  ์žˆ์Šต๋‹ˆ๋‹ค. ๊ฑด๋„ˆ๋›ฐ๋Š” ์—ฐ๊ฒฐ์€ ์‹ค์ œ ๋‡Œ์—์„œ ์˜๊ฐ์„ ์–ป์€ ๊ตฌ์กฐ๋กœ, ๋ ˆ์Šค๋„ท์˜ ๊นŠ์ด๋ฅผ ๋”ํ•˜๋Š” ๋ฐ ๋„์›€์„ ์ฃผ์—ˆ์Šต๋‹ˆ๋‹ค. (์ธต์„ ๋†’์ด ์Œ“์„ ์ˆ˜ ์žˆ๊ฒŒ ํ•ด์ฃผ์—ˆ์Šต๋‹ˆ๋‹ค.)

์™œ ์‹ ๊ฒฝ๋ง์ด ๊นŠ์œผ๋ฉด ์ข‹์€์ง€, ์™œ ๊ฑด๋„ˆ๋›ฐ๋Š” ์—ฐ๊ฒฐ์ด ๋„์›€์ด ๋˜๋Š”์ง€, ์ƒŒ๋“œ์œ„์น˜์˜ ๋น„์œ ๋กœ ์„ค๋ช…ํ•˜๊ธฐ์—๋Š” ํ•œ๊ณ„๊ฐ€ ์žˆ์–ด์„œ, ๋Œ€๋‹ต์€ ์ดํ›„ ๊ผญ์ง€๋“ค๋กœ ๋ฏธ๋ฃจ๋„๋ก ํ•˜๊ฒ ์Šต๋‹ˆ๋‹ค. ์—ฌ๊ธฐ์„œ๋Š” ๋ ˆ์Šค๋„ท์ด ๊ฑด๋„ˆ๋›ฐ๋Š” ์—ฐ๊ฒฐ์„ 2-3์ธต๋งˆ๋‹ค ๋ฐ˜๋ณตํ•ด์„œ ๊ฐ€์ง€๊ณ  ์žˆ์–ด์„œ, ์ƒŒ๋“œ์œ„์น˜๋ฅผ ์Œ“๋“ฏ์ด ๊ฐ™์€ ๊ตฌ์กฐ๋ฅผ ๋ฐ˜๋ณต๋งŒ ํ•˜๋ฉด ์‰ฝ๊ฒŒ ๊นŠ์–ด์ง€๋„๋ก ํ•  ์ˆ˜ ์žˆ๋‹ค๋Š” ์ ๋งŒ ์–ธ๊ธ‰ํ•˜๊ฒ ์Šต๋‹ˆ๋‹ค. (์›๋ž˜ ๋…ผ๋ฌธ์˜ ๊ทธ๋ฆผ 3์„ ์ฐธ๊ณ ํ•˜์„ธ์š”.)

๋‡Œ๊ณผํ•™/AI์— ๊ด€ํ•œ ์ฒซ ๊ผญ์ง€์˜€์Šต๋‹ˆ๋‹ค. ๋น„์œ ๊ฐ€ ์•„์ง ์–ด์ƒ‰ํ•˜์ง€๋งŒ, ๋” ์‹œ๋„ํ•˜๋ฉด์„œ ๊ฐœ์„ ํ•ด ๋ณด๊ฒ ์Šต๋‹ˆ๋‹ค. ์ ์–ด๋„ ๋‡Œ๊ณผํ•™์— ๊ด€ํ•ด ์จ ๋ณด๊ฒ ๋‹ค๋Š” ์ œ ์ƒˆํ•ด ๋‹ค์ง์„ ์ƒˆํ•ด ์ฒซ๋‚ ์— ์‹ค์ฒœํ•œ๋‹ค๋Š” ์˜๋ฏธ๋Š” ์žˆ๊ฒ ๋„ค์š”. ๐Ÿ™‚