Pro Masking Tutorial (using vector and alpha masking)

Other video games, TV shows, movies, general chit-chat...this is an all-purpose off-topic board where you can talk about anything that doesn't have its own dedicated section.

Pro Masking Tutorial (using vector and alpha masking)

Postby End Boss on Sat Jul 02, 2005 4:58 pm

::PROFESSIONAL MASKING TUTORIAL::

Hey, welcome to my first tutorial (hopefully not the last). I'm really happy with the creation of this new forum, finally I have a place that combines the 3 loves (basketball, photoshop, gaming) for me to call home. I'm really optimistic about learning with and teaching you guys.

I'm separating this tutorial into 2 parts, theory and practical. The theory side contains the explanations of the various concepts involved in what I consider to be professional masking (and hopefully isn't too long winded). I've tried to organise it in such a way that if you do have a particular question that you're able to find it easily without getting too bogged down in stuff you may already know. The practical aspect of the tutorial is where I'll give a detailed example and run through of the theory that I've covered. If you're familiar with the theory already or you feel happier to learn by example feel free to skip ahead to the practical section. I'm providing the theory part just in case you wanted to have a little more in depth understanding of the principles at work in these moderately advanced techniques.

If you have any questions about masking or you want to request another tutorial, feel free to private message me or post in this thread. I hope you pick up something that you can use. Enjoy!

::THEORY::

What is masking?
masking is the process of designating transparent (or semi-transparent) sections of bitmap layers. Masking is commonly used to make selections and remove unwanted backgrounds. There are 2 forms of masking (vector and alpha) which I consider to have the capacity to deliver professional results without fail.

What is professional masking?
From what I've gathered in bits and pieces it seems that the majority of people using PS here are using quick masking, the polygonal lasso, or the magnetic lasso. Those 3 tools all have their place in on the run, simplistic to intermediate detail selections. Where they fall short of meeting professional standards is in the ability to edit and manage those selections after the fact. Most of the time you'll be lucky (or smart) and have the transparency preserved by the layer itself and have the original background still intact in a hidden layer, but if you wanted to then change the boundries of that selection you'd have to make selections from the original, copy them, and merge them with the desired layer. Alpha and Vector masking allow full editing of the boundries of selection at any stage and that's where the whole process of selection becomes professional. Gone are the days of holding alt and shift to add and subtract from a selection. This is getting exactly what you want every time with the movement of a vertex or the stroke of a brush. The results are profound.

What are vector images?
Vector images are graphics that are created and edited by the manipulation of points, which when connected to each other form lines or areas of color. Vector points contain mathmatical information relating to the position, direction, and curve of the lines created when points are joined together. As such vector graphics are (virtually) infinitely scalable and are always expressed (in rasterized form) in the highest detail permitted by the display conditions. I could use the same vector graphic to print an area of a coin as I could the area of a bill board.

What are raster images?
Raster images are essentially the opposite of vector images. The graphical information is stored in pixels, where each pixel contains a set color and those pixels when displayed together adjacently form raster images that usually have inherent meaning (whether it's your signature or a picture of boobies). All images you see on the web (other than flash animations) are rasterized images. By their very nature raster images only contain a set amount of information, meaning that if they're scaled up they only contain the amount of detail expressed in the smallest rendering of that image. Likewise graphical information is lost when you scale raster images down.

What is vertor masking?
Vector masking is the creation of transparent areas of a layer using the pen tool to establish the boundries of those transparent regions. Vectors, as opposed rasterized images, are composed of free moving points (known as vertices or points) that contain information relating to both position and direction. This information, unlike it's rasterized brethren, can be scaled without loss of information and is always rendered in the highest possible detail as determined by the display area and screen resolution (hence doesn't experience pixelation). Basically what this means is that you can create a smooth vector boundry at high levels of magnification and photoshop will automatically anti-alias the smoothest selection given the pixel information contained in the raster image.

Where this all comes into it's own as a professional means of selection is that the designated transparent areas of the vector mask can be rolled back and displayed at any time by moving the vertices and altering their directional handles. In all my years with photoshop (since '98 :cool: ) using combinations of lassos, smudge/blur tools, quick masking, and even the paint brush I've never found anything that can produce as clean a selection as vector masking. If you're about to use this technique for the first time, you'll jump for joy when you find out how clean, easy, and powerful this is.

Pros/Cons of Vector Masking
+ produces the best solid edge smooth curve masking possible bar none
+ easy to edit and maintain
+ excellent for selecting large areas and groups of areas at a time
- not good for creating transparent maps of highly detailed areas (such as hair)
- not good for selecting areas with varying or graded levels of opacity/transparency
- takes a bit of getting used to (but that passes quickly.. you'll be using the pen tool like a pro soon enough)

What is alpha masking?
Alpha masking is a technique that allows you designate transparent areas of a layer by filling in the areas you wish to become transparent with any of the tools that is conventually used to create solid graphics (brush, pen, paint bucket, etc). Probably the best way to describe it is that it's like drawing in reverse where you start with an image and paint away the parts you don't want. It's virtually the same as using the eraser, but with the notable exception that you're never deleting any of the graphical information contained within the unmasked image. By using the eraser whilst alpha masking you actually undo the "erasing" that you've already done. It's very similar in principle and practice to quick masking, except that you can see the results in real time (which is a lot of fun btw), and that in the sense that it's not a conventional selection it doesn't just dissapear when you start doing something else.

Pros/Cons of Alpha Masking
+ excellent for quickly adding masks for sections with fine detail
+ more free form and generally less tedious to use than vector masking
+ the only real way to achieve variation in opacity/transparency in masking (outside of quick masking, but alpha masking is basically quick masking with added managability)
- is tedious when using for large selections (as you either have to paint or fill all sections)
- much harder to produce the same solid boundry smooth line results as vector masking

When should I use vector/alpha masking?
By themselves vector and alpha masking are tremendously powerful, fun, and open up a world of possibilities in and of themselves, but it's when you team these 2 together that flood gates really burst open. Both techniques have their inherent strengths and weaknesses, but luckily for us those strengths and weaknesses line up so that when used together, vector and alpha masking complement each other perfectly.

For anything with solid lines, curves, and that require a crisp result, you should immediately think vector masking. Just thinking of the effort I used to expend aiming for some small semblance of the results made possible by vector masking is enough to make me cry.

For anything with fuzzy edges, varying levels of transparency, smudging or blurring, immediately think alpha masking.

The best part of it all is how easy it is to use these two techniques in conjunction to mask the same layer, as you'll quickly find out in the practical section of this tutorial. Thus concludes the theory component of this tutorial (for now).

::PRACTICAL::

Okay.. Beginning the Practical component at 12:30 PM (GMT + 10). I imagine this is going to take the vast majority of time on the documentation side of things but rest assured that however long I've taken overall that only 10 to 20 mins has been used on performing the vector and alpha masking, which should probably tell you just how slow a writer I am. I'm going to update as I go, so if nething goes majorly wrong I lose very little. Here we go.

1. Getting Started
I always like to start with a bit of house keeping, making things organised from the get go even though we'll probably only be using the 2 layers. Out of habit I whenever I'm editing a layer I always keep the original in a seperate layer, for our purposes you'll find that having the duplicate layer will come in handy should things get fiddly as they can when you have subtle variation between foreground and background. For the duration of this practical component I'm going to write anything I do in photoshop (no matter how slight, unless repeated) in colored text here. So to kick things off.

Image

a. Right click on Stevey directly above and select copy

b. Switch to PhotoShop and press CTRL + N to create a new document

The dimensions of the image on the clipboard will always be adopted as the dimensions for the new document in photoshop.

c. Name the document "nash_masked", make sure that the Background Contents option is set to "Transparent" and click ok

d. Press CTRL + V twice to paste 2 layers of the nash image onto the canvas

e. Select the layers tab (by pressing F7) and rename Layer 1 to "original" and Layer 2 to "masked"

2. The Pen Tool
In step 2 we'll be creating a vector outline of nash using the pen tool. I'll do my best explain all of the tools relating to vectors in PS and go through their uses, but if you're able to do things right in the first go round you'll overwhelmingly just be using the pen tool and the direct selection tool (for minor adjustments). Like I say though I'll try and work the others in there just for the sake of completeness of this tutorial. It just occured to me that up until now I haven't mentioned that in PS vector drawings are known as paths. Paths simply refer to any vector information currently in existence on the canvas. There is a paths tab that keeps track of all the paths ( :P ), which I always keep mounted in the mounting bay at the top right hand corner of the PS window.

a. Select the masked layer from the layers window, then select the pen tool from the tool box (or by pressing P)

Even though paths exist independently of layers, I selected the masked layer purely to maintain a sense of mental organisation.

b. Open the Paths window by selecting it from the Window menu

If you don't already have the paths window mounted I suggest you do it at this stage, once again for the sake of organisation and good form.

c. In the paths window create a new path by clicking the little universal new icon in the bottom right corner of the window next to the trash icon and name the new path "nash mask"

When you begin using the pen tool it will automatically add any paths you make to "Work Path", but once you use that for selection or delete it work path is emptied and you lose that path forever. When you use the path to create a vector mask the path information will be stored in a vector mask path so you can get away with never using the features in paths window if you're confident. I'm trying to encourage good technique here, and I can tell you from experience that you can get burned being lax on these fine details that really take only a matter of seconds.

d. With the pen tool selected plot the first point (or vertex) at the base of Nash's right shoe (our left). I'm doing this at 400% magnification (zoom in and out by holding ALT + MOUSEWHEEL UP/DOWN or CTRL + -/+)

fig 2d
Image

As far as I know there's no right or wrong place to start just so long as you start at the midpoint of a curve or a corner. If you start in the middle of a straight line then you've basically created an unnecessary vertex. Not a big deal by any means, but worthy of mentioning none the less.

Time Out: T1. Adding curve to paths using Directional Handlebars

Before we plot the next point in our path I'd like to explain how we create curved lines with paths. If you're familiar with the pen tool and all of it's subtleties, then feel free to skip ahead. Bear in mind, however, that the pen tool is moderately advanced and I certaintly didn't grasp all of it's intricacies immediately, so even if you have experience with the pen tool you may still learn something that could significantly improve the efficiency with which you use it.

There are 4 basic types of line: straight lines, symmetrically curved lines, asymmetrically curved lines, and hybrid lines.

The basic underlying principle behind the creation of all types of lines is if you create a point by clicking once you define no incoming or outgoing curve. However, if you click and drag then release the left mouse button that point will then effect the incoming and outgoing curve of the lines it creates by joining with other points.


fig t1_a
Image

straight lines
straight lines are achieved by creating one point (clicking the pen tool once without dragging to define a point with no incoming or outgoing curve) and then creating another point by clicking the pen tool once again (elsewhere). The resulting line created between the two point will be straight.


fig t1_b (animated)
Image

A straight line will always be created between 2 points that don't define incoming or outgoing curves, but that doesn't mean that the adjacent connected lines won't be curved.

fig t1_c
Image



Last edited by End Boss on Sun Jul 03, 2005 5:51 pm, edited 13 times in total.
End Boss - End Boss - The Universe and all that lies therein
Image
User avatar
End Boss
 
Posts: 755
Joined: Wed Oct 15, 2003 7:23 pm
Location: Melbourne, Australia

Postby Mentally Hilarious on Sat Jul 02, 2005 5:21 pm

For gods sake sticky this. Brilliant tut Harvey.
Image
User avatar
Mentally Hilarious
 
Posts: 1045
Joined: Mon Dec 27, 2004 6:07 am

Postby End Boss on Sat Jul 02, 2005 5:39 pm

Image

just wanted to present this as something approaching finished.. atleast the masking aspect of things.. don't want to use this one as a tutorial (although it's a perfect example) because I've already done it.. like to keep moving forward
End Boss - End Boss - The Universe and all that lies therein
Image
User avatar
End Boss
 
Posts: 755
Joined: Wed Oct 15, 2003 7:23 pm
Location: Melbourne, Australia

Postby FendeR` on Sat Jul 02, 2005 11:01 pm

Is this OK?

Image
FendeR` - Captain
Image
User avatar
FendeR`
 
Posts: 1763
Joined: Sun Feb 06, 2005 2:50 am
Location: Pennsylvania

Postby J@3 on Sat Jul 02, 2005 11:11 pm

Don't understand a word of it but I'll sticky it nonetheless.
User avatar
J@3
 
Posts: 19815
Joined: Thu Mar 11, 2004 3:25 pm
Location: MLB

Postby FendeR` on Sat Jul 02, 2005 11:11 pm

Jae wrote:Don't understand a word of it but I'll sticky it nonetheless.


lol that's what I thought. Now I sort of get the idea I think, I'll just wait until he finishes it because I have an idea of what he means.
FendeR` - Captain
Image
User avatar
FendeR`
 
Posts: 1763
Joined: Sun Feb 06, 2005 2:50 am
Location: Pennsylvania

Postby End Boss on Sun Jul 03, 2005 6:42 am

fender, that should be fine.. I'm happy to do multiple examples.. I'll make a start on that today

if there's anything anyone wants explained more plainly or in further detail don't hesitate to ask. I'd rather create a resource that speaks plainly and people can readily use than something that sounds sophisticated but isn't as easily understood. I'm sure the practical component of the tutorial will help clear things up, but like I say if there's anything you want explained then I'm only too happy to help.

UPDATE

sorry.. been very busy lately.. will finish this as soon as possible..

today I used these techniques to create t-shirt transfers for T-Shirts (for my bro's girlfriend who is going to give it to her bro as a present) with pretty damn fine results.. I've had to reduce the sizes and such (because of the restrictions imposed on this forum), so they don't really show off how good the masking was :( .. but I thought I'd post them anyway

Image

Image

sorry for the file size of the 2nd image (113k).. just wasnt worth showing in any lower color detail.. guess I could've posted it as a jpg with a white background
End Boss - End Boss - The Universe and all that lies therein
Image
User avatar
End Boss
 
Posts: 755
Joined: Wed Oct 15, 2003 7:23 pm
Location: Melbourne, Australia

Postby nbalive744 on Fri Sep 09, 2005 7:32 am

i am looking up masking on good-tutorials.com right now 8-) (Y)
Image
Click on the Sig for my PS Thread, Thanks Colin
New York Knicks Announcer during a Free Throw Attempt wrote:And you see the fans waving those thundersticks, they only work if you can't shoot
User avatar
nbalive744
 
Posts: 2147
Joined: Thu Jul 21, 2005 7:15 am
Location: PA

Postby Colin on Fri Sep 09, 2005 10:16 am

You're not going to find anything more usuful than this on good-tutorials.com. You'll have to filter through tuts that tell you to use polygonal lasso.
C#
Image
Pretty Flaco
User avatar
Colin
 
Posts: 5913
Joined: Thu Mar 06, 2003 7:02 am
Location: Van-City

Postby Colin on Mon Sep 26, 2005 8:57 am

Everyone look at this. If you don't know how to cut out an image properly carefully read this tutorial. You really need to.

And I would really like it if you could explain, in at least a bit of detail, your method of alpha masking. A full-on tut on that may be time-consuming, but I would love just a few bits of info.
C#
Image
Pretty Flaco
User avatar
Colin
 
Posts: 5913
Joined: Thu Mar 06, 2003 7:02 am
Location: Van-City

Postby nbalive744 on Mon Sep 26, 2005 9:14 am

yea me too, that would be cool (Y)
Image
Click on the Sig for my PS Thread, Thanks Colin
New York Knicks Announcer during a Free Throw Attempt wrote:And you see the fans waving those thundersticks, they only work if you can't shoot
User avatar
nbalive744
 
Posts: 2147
Joined: Thu Jul 21, 2005 7:15 am
Location: PA

Postby End Boss on Mon Sep 26, 2005 9:30 am

definately my plan to finish the tutorial and write many more.. but atm juggling my time between uni, pulling together a folio, and managing the fantasy league.. but I'll give a little explanation on alpha masking and how you might use it for hair to tide you over.. gimme a tic
End Boss - End Boss - The Universe and all that lies therein
Image
User avatar
End Boss
 
Posts: 755
Joined: Wed Oct 15, 2003 7:23 pm
Location: Melbourne, Australia

Postby BIG GREEN on Mon Sep 26, 2005 11:09 am

This is very useful...i consider myself to be "ok" with photoshop..when it comes to patching and stuff...but i dont know how to do this. I'm gonna apply this to patching though. :)
Image
A big fan of the emerald hue and much higher state of being/
Yohance "thug" Bailey on the scene...now known as Big Green/
User avatar
BIG GREEN
 
Posts: 4413
Joined: Thu Sep 19, 2002 1:18 pm
Location: Bronx, New york

Postby End Boss on Mon Sep 26, 2005 11:23 am

you don't need to sticky the thread there's a tutorials thread stickied that links it (Y)
End Boss - End Boss - The Universe and all that lies therein
Image
User avatar
End Boss
 
Posts: 755
Joined: Wed Oct 15, 2003 7:23 pm
Location: Melbourne, Australia

Postby adz192 on Fri Sep 01, 2006 2:27 pm

Hey End Boss, I've got a question..

How can I copy the masked pic?
This was the result:
Image
Image
Pambato ng mga taga-Cotabato!
Ya heard it right mga BabeSen!
User avatar
adz192
 
Posts: 61
Joined: Sat Mar 25, 2006 9:38 am
Location: Cotabato, Philippines


Return to Off-Topic

Who is online

Users browsing this forum: No registered users and 7 guests